MSTRE-Net: Multistreaming Acoustic Modeling for Automatic Lyrics Transcription

TL;DR

This paper introduces MSTRE-Net, a multistreaming neural network architecture for automatic lyrics transcription, along with novel preprocessing techniques, achieving state-of-the-art results on a new, diverse dataset.

Contribution

The paper presents MSTRE-Net, a novel multistreaming neural network architecture, and new preprocessing methods for improved automatic lyrics transcription.

Findings

MSTRE-Net outperforms previous models in recognition accuracy.

Using diverse training data improves model robustness.

The new dataset enhances evaluation of ALT systems.

Abstract

This paper makes several contributions to automatic lyrics transcription (ALT) research. Our main contribution is a novel variant of the Multistreaming Time-Delay Neural Network (MTDNN) architecture, called MSTRE-Net, which processes the temporal information using multiple streams in parallel with varying resolutions keeping the network more compact, and thus with a faster inference and an improved recognition rate than having identical TDNN streams. In addition, two novel preprocessing steps prior to training the acoustic model are proposed. First, we suggest using recordings from both monophonic and polyphonic domains during training the acoustic model. Second, we tag monophonic and polyphonic recordings with distinct labels for discriminating non-vocal silence and music instances during alignment. Moreover, we present a new test set with a considerably larger size and a higher musical variability compared to the existing datasets used in ALT literature, while maintaining the gender balance of the singers. Our best performing model sets the state-of-the-art in lyrics transcription by a large margin. For reproducibility, we publicly share the identifiers to retrieve the data used in this paper.