TouchFusion: Multimodal Wristband Sensing for Ubiquitous Touch Interactions

TL;DR

TouchFusion introduces a multimodal wristband that enables versatile, surface-agnostic touch interactions without additional hardware, validated on a large participant dataset for robust performance in adaptive interfaces.

Contribution

It presents a novel multimodal sensing wristband combining multiple sensing modalities for stateful touch detection and interaction on various surfaces, with extensive validation on a large participant pool.

Findings

Enables touch detection on environmental and body surfaces.

Supports simple gestures and tracking for adaptive interfaces.

Validated on 100 participants, outperforming typical wearable studies.

Abstract

TouchFusion is a wristband that enables touch interactions on nearby surfaces without any additional instrumentation or computer vision. TouchFusion combines surface electromyography (sEMG), bioimpedance, inertial, and optical sensing to capture multiple facets of hand activity during touch interactions. Through a combination of early and late fusion, TouchFusion enables stateful touch detection on both environmental and body surfaces, simple surface gestures, and tracking functionality for contextually adaptive interfaces as well as basic trackpad-like interactions. We validate our approach on a dataset of 100 participants, significantly exceeding the population size of typical wearable sensing studies to capture a wider variance of wrist anatomies, skin conductivities, and behavioral patterns. We show that TouchFusion can enable several common touch interaction tasks. Using TouchFusion, a wearer can summon a trackpad on any surface, control contextually adaptive interfaces based on where they tap, or use their palm as an always-available touch surface. When paired with smart glasses or augmented reality devices, TouchFusion enables a ubiquitous, contextually adaptive interaction model.