Project Description

Real-Time Capture of Holistic Tangible Interactions

When digital applications aim to blend virtual and real worlds, understanding the actual physical actions of users becomes an important task; the precise timing of these tangible interaction events is needed, along with the identity, and possibly location and history, of all involved actors/objects. With multiple actors or objects, it is difficult to identify who touches which object and when. Instrumenting objects for Body Channel Communication (BCC) allows message exchange around the human body between instrumented objects and the user themselves. In this paper we show how BCC can be utilized to perform under real-time conditions so that we can directly notice touch events (and the identity of actors). TangibleID is a framework that unifies tangible interaction capture for objects and users based on wearable BCC. TangibleID provides identification and communication with tagged objects/users in less than 120 ms and supports a variety of tangible interactions, without the need to restrict user (hand) movements or to maintain line-of-sight connection to cameras. When an AR application is combined with TangibleID, a new tangible mixed reality experience is achieved, as demonstrated in the “Haunted Castle” showcase. The paper presents an end-to-end technical evaluation including trade-offs regarding robustness and speed of touch recognition, outlines the breadth of interaction modalities, and reports on an initial user assessment.


Enabling Interactive Infrastructure with Body Channel Communication

Body channel communication (BCC) uses the human body to carry signals, and therefore provides communication and localization that are directly tied to human presence and actions. Previous BCC systems were expensive, could operate only in a laboratory, or only focused on special use cases. We present here an end-to-end BCC system that is designed for ambient intelligence. We introduce the BCC infrastructure that consists of portable devices (e.g., a simple sphere), mobile devices (e.g., a smartwatch-like wristband), and stationary devices (e.g., floor/wall tiles). We also describe the core technology that is used in each of these units. The TouchCom hardware-software platform is a simple transceiver with software-centered processing. The focus on software (even the implementation of the physical layer is based on software) allows the adaptivity that is necessary to operate a BCC-based system in practice. The paper describes the design and a prototype implementation of the TouchCom-based interactive infrastructure and provides evidence that this BCC infrastructure works for different persons and different setups. The system provides moderate bandwidth (about 3.5 kb/s) that is suitable for several usage scenarios like games, localization, and identification. The implemented demonstrations illustrate the benefits these applications gain when touching an object is tied to communication.


Designing Groundless Body Channel Communication Systems: Performance and Implications

Novel interactions that capacitively couple electromagnetic (EM) fields between devices and the human body are gaining more attention in the human-computer interaction community. One class of these techniques is Body Channel Communication (BCC), a method that overlays physical touch with digital information. Despite the number of published capacitive sensing and communication prototypes, there exists no guideline on how to design such hardware or what are the application limitations and possibilities. Specifically, wearable (groundless) BCC has been proven in the past to be extremely challenging to implement. Additionally, the exact behavior of the human body as an EM-field medium is still not fully understood today. Consequently, the application domain of BCC technology could not be fully explored. This paper addresses this problem. Based on a recently published general purpose wearable BCC system, we first present a thorough evaluation of the impact of various technical parameter choices and an exhaustive channel characterization of the human body as a host for BCC. Second, we discuss the implications of these results for the application design space and present guidelines for future wearable BCC systems and their applications. Third, we point out an important observation of the measurements, namely that BCC can employ the whole body as user interface (and not just hands or feet). We sketch several applications with these novel interaction modalities.



Related Publication

Related Projects

Thumb EM-Sense

EMI based Sensing and Communication

By exploiting the unintentional electromagnetic (EM) noise emitted by many everyday electro-mechanical objects it is possible to robustly classify the device by type and determine its individual identity, owing to their different internal operation and variations in manufacturing. Furthermore, when properly modulated these electromagnetic emissions can be used as an untapped communication channel capable of transmitting arbitrary data through user touch.

3 Award
3 Papers
2 Video
10 Featured Press


This project originated as a collaboration between Disney Research and ETH Zurich with follow on research efforts at the University of Michigan.