Multi-Convformer: Extending Conformer with Multiple Convolution Kernels

TL;DR

Multi-Convformer enhances Conformer models by integrating multiple convolution kernels with gating, improving local context modeling and achieving up to 8% WER reduction across various datasets.

Contribution

Introduces Multi-Convformer, a novel convolution module with multiple kernels and gating, offering better local dependency modeling and parameter efficiency.

Findings

Achieves up to 8% relative WER reduction.

Performs competitively with Conformer variants like CgMLP and E-Branchformer.

Demonstrates effectiveness across four datasets and three modeling paradigms.

Abstract

Convolutions have become essential in state-of-the-art end-to-end Automatic Speech Recognition~(ASR) systems due to their efficient modelling of local context. Notably, its use in Conformers has led to superior performance compared to vanilla Transformer-based ASR systems. While components other than the convolution module in the Conformer have been reexamined, altering the convolution module itself has been far less explored. Towards this, we introduce Multi-Convformer that uses multiple convolution kernels within the convolution module of the Conformer in conjunction with gating. This helps in improved modeling of local dependencies at varying granularities. Our model rivals existing Conformer variants such as CgMLP and E-Branchformer in performance, while being more parameter efficient. We empirically compare our approach with Conformer and its variants across four different datasets and three different modelling paradigms and show up to 8% relative word error rate~(WER) improvements.