E-Branchformer: Branchformer with Enhanced merging for speech recognition

TL;DR

E-Branchformer improves upon Branchformer by enhancing the merging process and stacking modules, achieving state-of-the-art speech recognition performance on LibriSpeech without external data.

Contribution

The paper introduces E-Branchformer, a novel model that enhances Branchformer with better merging techniques and additional modules for improved ASR accuracy.

Findings

Achieves new state-of-the-art WERs of 1.81% and 3.65% on LibriSpeech test sets.

Outperforms previous models without external training data.

Demonstrates the effectiveness of enhanced merging and stacking in speech recognition.

Abstract

Conformer, combining convolution and self-attention sequentially to capture both local and global information, has shown remarkable performance and is currently regarded as the state-of-the-art for automatic speech recognition (ASR). Several other studies have explored integrating convolution and self-attention but they have not managed to match Conformer's performance. The recently introduced Branchformer achieves comparable performance to Conformer by using dedicated branches of convolution and self-attention and merging local and global context from each branch. In this paper, we propose E-Branchformer, which enhances Branchformer by applying an effective merging method and stacking additional point-wise modules. E-Branchformer sets new state-of-the-art word error rates (WERs) 1.81% and 3.65% on LibriSpeech test-clean and test-other sets without using any external training data.