Poster: Recognizing Hidden-in-the-Ear Private Key for Reliable Silent Speech Interface Using Multi-Task Learning

TL;DR

This paper introduces HEar-ID, a silent speech interface system using earbuds that combines speech recognition and user authentication through multi-task learning, enabling reliable silent spelling and secure user verification.

Contribution

The novel integration of silent speech recognition and user authentication in a single model using multi-task learning on commodity earbuds.

Findings

Achieves accurate silent spelling of 50 words

Provides robust user authentication rejecting impostors

Operates effectively on consumer-grade earbuds

Abstract

Silent speech interface (SSI) enables hands-free input without audible vocalization, but most SSI systems do not verify speaker identity. We present HEar-ID, which uses consumer active noise-canceling earbuds to capture low-frequency "whisper" audio and high-frequency ultrasonic reflections. Features from both streams pass through a shared encoder, producing embeddings that feed a contrastive branch for user authentication and an SSI head for silent spelling recognition. This design supports decoding of 50 words while reliably rejecting impostors, all on commodity earbuds with a single model. Experiments demonstrate that HEar-ID achieves strong spelling accuracy and robust authentication.