TeethTap: Recognizing Discrete Teeth Gestures Using Motion and Acoustic Sensing on an Earpiece

TL;DR

TeethTap is a novel hands-free, eyes-free input method using an earpiece with sensors to recognize 13 teeth gestures with over 90% accuracy, suitable for accessibility and various real-world activities.

Contribution

This paper introduces TeethTap, a new wearable system that combines motion and acoustic sensing to accurately recognize discrete teeth gestures in real-time.

Findings

Achieved 90.9% classification accuracy in laboratory tests.

Identified accuracy differences between single-side and dual-side sensors.

Demonstrated gesture recognition during activities like eating, speaking, walking, and jumping.

Abstract

Teeth gestures become an alternative input modality for different situations and accessibility purposes. In this paper, we present TeethTap, a novel eyes-free and hands-free input technique, which can recognize up to 13 discrete teeth tapping gestures. TeethTap adopts a wearable 3D printed earpiece with an IMU sensor and a contact microphone behind both ears, which works in tandem to detect jaw movement and sound data, respectively. TeethTap uses a support vector machine to classify gestures from noise by fusing acoustic and motion data, and implements K-Nearest-Neighbor (KNN) with a Dynamic Time Warping (DTW) distance measurement using motion data for gesture classification. A user study with 11 participants demonstrated that TeethTap could recognize 13 gestures with a real-time classification accuracy of 90.9% in a laboratory environment. We further uncovered the accuracy differences on different teeth gestures when having sensors on single vs. both sides. Moreover, we explored the activation gesture under real-world environments, including eating, speaking, walking and jumping. Based on our findings, we further discussed potential applications and practical challenges of integrating TeethTap into future devices.