SCaLa: Supervised Contrastive Learning for End-to-End Speech Recognition

TL;DR

This paper introduces SCaLa, a supervised contrastive learning framework that improves phonemic representation in end-to-end speech recognition, reducing errors by leveraging phoneme boundaries for better feature learning.

Contribution

The paper extends self-supervised contrastive coding to a supervised setting using phoneme boundaries, enhancing phonemic discrimination in ASR models.

Findings

Achieved 2.8 and 1.4 points CER reduction on reading and spontaneous speech datasets.

Utilized phoneme forced-alignment to guide contrastive learning, improving recognition accuracy.

Demonstrated effectiveness of supervised contrastive learning in end-to-end ASR systems.

Abstract

End-to-end Automatic Speech Recognition (ASR) models are usually trained to optimize the loss of the whole token sequence, while neglecting explicit phonemic-granularity supervision. This could result in recognition errors due to similar-phoneme confusion or phoneme reduction. To alleviate this problem, we propose a novel framework based on Supervised Contrastive Learning (SCaLa) to enhance phonemic representation learning for end-to-end ASR systems. Specifically, we extend the self-supervised Masked Contrastive Predictive Coding (MCPC) to a fully-supervised setting, where the supervision is applied in the following way. First, SCaLa masks variable-length encoder features according to phoneme boundaries given phoneme forced-alignment extracted from a pre-trained acoustic model; it then predicts the masked features via contrastive learning. The forced-alignment can provide phoneme labels to mitigate the noise introduced by positive-negative pairs in self-supervised MCPC. Experiments on reading and spontaneous speech datasets show that our proposed approach achieves 2.8 and 1.4 points Character Error Rate (CER) absolute reductions compared to the baseline, respectively.