PI-Whisper: Designing an Adaptive and Incremental Automatic Speech Recognition System for Edge Devices

TL;DR

PI-Whisper is an adaptive, incremental speech recognition system designed for resource-limited edge devices, improving accuracy and fairness across diverse speakers without retraining.

Contribution

The paper introduces PI-Whisper, a novel ASR system that adaptively and incrementally improves recognition capabilities on edge devices, addressing fairness and resource constraints.

Findings

Achieves up to 13.7% relative WER reduction compared to baselines.

Supports incremental adaptation without retraining.

Scales linearly with computing resources.

Abstract

Edge-based automatic speech recognition (ASR) technologies are increasingly prevalent in the development of intelligent and personalized assistants. However, resource-constrained ASR models face significant challenges in adaptivity, incrementality, and inclusivity when faced with a diverse population. To tackle those challenges, we propose PI-Whisper, a novel ASR system that adaptively enhances recognition capabilities by identifying speakers' characteristics in real-time. In this work, we show how the design of PI-Whisper allows for incremental adaptation of new characteristics without the need for repetitive retraining, enhances recognition capabilities, and improves equity and fairness across diverse speaker groups. PI-Whisper demonstrates these advantages by achieving state-of-the-art accuracy, reducing the word error rate (WER) by up to 13.7% relative to baselines while scaling linearly to computing resources.