ACES: Accent Subspaces for Coupling, Explanations, and Stress-Testing in Automatic Speech Recognition

TL;DR

This paper introduces ACES, a method to identify and analyze accent-related features in speech recognition models, revealing deep entanglement between accent and recognition features and providing a new fairness auditing tool.

Contribution

We propose ACES, a three-stage framework for extracting and testing accent-discriminative subspaces in ASR representations, advancing fairness analysis in speech recognition.

Findings

Accent subspaces significantly affect WER disparities.

Removing accent subspaces worsens recognition performance.

Accent features are deeply entangled with recognition-critical features.

Abstract

ASR systems exhibit persistent performance disparities across accents, but whether these gaps reflect superficial biases or deep structural vulnerabilities remains unclear. We introduce ACES, a three-stage audit that extracts accent-discriminative subspaces from ASR representations, constrains adversarial attacks to them, and tests whether removing them improves fairness. On Wav2Vec2-base with seven accents, imperceptible perturbations (~60 dB SNR) along the accent subspace amplify the WER disparity gap by nearly 50% (21.3->31.8 pp), exceeding random-subspace controls; a permuted-label test confirms specificity to genuine accent structure. Partially removing the subspace worsens both WER and disparity, revealing that accent-discriminative and recognition-critical features are deeply entangled. ACES thus positions accent subspaces as powerful fairness-auditing tools, not simple erasure levers.