M-CIF: Multi-Scale Alignment For CIF-Based Non-Autoregressive ASR

TL;DR

This paper introduces M-CIF, a multi-scale alignment method for non-autoregressive speech recognition that improves stability and accuracy by integrating character and phoneme supervision, leading to significant WER reductions.

Contribution

The paper proposes Multi-scale CIF (M-CIF), a novel multi-level alignment approach that enhances the stability and performance of CIF-based NAR ASR models across languages.

Findings

M-CIF reduces WER by over 4% in German and French on CommonVoice.

Phoneme and character layers are crucial for improved alignment.

Analysis metrics show better phonetic and segmentation accuracy with M-CIF.

Abstract

The Continuous Integrate-and-Fire (CIF) mechanism provides effective alignment for non-autoregressive (NAR) speech recognition. This mechanism creates a smooth and monotonic mapping from acoustic features to target tokens, achieving performance on Mandarin competitive with other NAR approaches. However, without finer-grained guidance, its stability degrades in some languages such as English and French. In this paper, we propose Multi-scale CIF (M-CIF), which performs multi-level alignment by integrating character and phoneme level supervision progressively distilled into subword representations, thereby enhancing robust acoustic-text alignment. Experiments show that M-CIF reduces WER compared to the Paraformer baseline, especially on CommonVoice by 4.21% in German and 3.05% in French. To further investigate these gains, we define phonetic confusion errors (PE) and space-related segmentation errors (SE) as evaluation metrics. Analysis of these metrics across different M-CIF settings reveals that the phoneme and character layers are essential for enhancing progressive CIF alignment.