VoiceAlign: A Shimming Layer for Enhancing the Usability of Legacy Voice User Interface Systems

TL;DR

VoiceAlign is a novel adaptive layer that enhances legacy voice user interfaces by transforming natural commands into system-compatible syntax, significantly improving usability and reducing errors without modifying existing systems.

Contribution

We introduce VoiceAlign, a shimming layer that uses AI to adapt natural voice commands for legacy VUIs, improving interaction success and user experience.

Findings

Reduced command failures by 50%

Decreased commands needed per task by 25%

Achieved over 90% accuracy with a small, locally fine-tuned language model

Abstract

Voice user interfaces (VUIs) are rapidly transitioning from accessibility features to mainstream interaction modalities. Yet most operating systems' built-in voice commands remain underutilized despite possessing robust technical capabilities. Through our analysis of four commercial VUI systems and a formative study with 16 participants, we found that fixed command formats require exact phrasing, restrictive timeout mechanisms discard input during planning pauses, and insufficient feedback hampers multi-step interactions. To address these challenges, we developed VoiceAlign, an adaptive shimming layer that mediates between users and legacy VUI systems. VoiceAlign intercepts natural voice commands, transforms them to match the required syntax using a large language model, and transmits these adapted commands through a virtual audio channel that remains transparent to the underlying system. In our evaluation with 12 participants, VoiceAlign reduced command failures by half, required 25% fewer commands per task, and significantly lowered cognitive and temporal demands when paired with an existing legacy VUI system. Furthermore, we created a synthetic dataset informed by our studies and fine-tuned a small language model that achieves over 90% accuracy with 200 ms response time when served locally, eliminating dependence on third-party APIs while enabling real-time interaction on edge devices. This work demonstrates how modern AI techniques can unlock the underutilized potential of legacy VUI systems without requiring system modifications, offering a practical solution without replacing existing infrastructure.