Lightweight End-to-End Speech Recognition from Raw Audio Data Using Sinc-Convolutions

TL;DR

This paper introduces Lightweight Sinc-Convolutions, a learnable feature extraction method integrated into end-to-end speech recognition models, achieving high accuracy with significantly reduced model size.

Contribution

It proposes a novel low-parameter Sinc-Convolution based feature extractor for end-to-end ASR, improving efficiency and accuracy over traditional handcrafted features.

Findings

Achieved 10.7% WER on TEDlium v2, outperforming log-mel filterbank features.

Model size is only 21% of the comparable architecture with traditional features.

Smooth convergence behavior enhanced by SpecAugment in time domain.

Abstract

Many end-to-end Automatic Speech Recognition (ASR) systems still rely on pre-processed frequency-domain features that are handcrafted to emulate the human hearing. Our work is motivated by recent advances in integrated learnable feature extraction. For this, we propose Lightweight Sinc-Convolutions (LSC) that integrate Sinc-convolutions with depthwise convolutions as a low-parameter machine-learnable feature extraction for end-to-end ASR systems. We integrated LSC into the hybrid CTC/attention architecture for evaluation. The resulting end-to-end model shows smooth convergence behaviour that is further improved by applying SpecAugment in time-domain. We also discuss filter-level improvements, such as using log-compression as activation function. Our model achieves a word error rate of 10.7% on the TEDlium v2 test dataset, surpassing the corresponding architecture with log-mel filterbank features by an absolute 1.9%, but only has 21% of its model size.