Contrastive Regularization for Accent-Robust ASR

TL;DR

This paper proposes supervised contrastive learning as a simple, effective regularization method to improve accent robustness in speech recognition systems, reducing error rates on unseen accents.

Contribution

It introduces a contrastive regularization technique for CTC fine-tuning that enhances accent invariance without changing model architecture or requiring explicit accent labels.

Findings

Achieves up to 29% relative WER reduction on unseen accents.

Promotes more compact and stable encoder representations.

Effective across multiple pretrained encoder models.

Abstract

ASR systems based on self-supervised acoustic pretraining and CTC fine-tuning achieve strong performance on native speech but remain sensitive to accent variability. We investigate supervised contrastive learning (SupCon) as a lightweight, accent-invariant auxiliary objective for CTC fine-tuning. An utterance-level contrastive loss regularizes encoder representations without architectural modification or explicit accent supervision. Experiments on the L2-ARCTIC benchmark show consistent WER reductions across multiple pretrained encoders, with up to 25 -- 29\% relative reduction under unseen-accent evaluation. Analysis using within-transcript cosine dispersion indicates that SupCon promotes more compact and stable representation geometry under accent variability. Overall, SupCon provides an effective and model-agnostic regularization strategy for improving accent robustness.