MuSLCAT: Multi-Scale Multi-Level Convolutional Attention Transformer for Discriminative Music Modeling on Raw Waveforms

TL;DR

This paper introduces MuSLCAT, a novel multi-scale, multi-level convolutional attention transformer architecture for raw waveform music modeling, demonstrating improved efficiency and competitive performance on music tagging and genre recognition tasks.

Contribution

The paper proposes MuSLCAT and MuSLCAN architectures that effectively model hierarchical and sequential music features directly from raw waveforms, with a lightweight design and novel attention mechanisms.

Findings

MuSLCAT and MuSLCAN outperform state-of-the-art models on benchmark datasets.

Both architectures achieve competitive results with fewer parameters.

The models effectively capture multi-scale and multi-level music features.

Abstract

In this work, we aim to improve the expressive capacity of waveform-based discriminative music networks by modeling both sequential (temporal) and hierarchical information in an efficient end-to-end architecture. We present MuSLCAT, or Multi-scale and Multi-level Convolutional Attention Transformer, a novel architecture for learning robust representations of complex music tags directly from raw waveform recordings. We also introduce a lightweight variant of MuSLCAT called MuSLCAN, short for Multi-scale and Multi-level Convolutional Attention Network. Both MuSLCAT and MuSLCAN model features from multiple scales and levels by integrating a frontend-backend architecture. The frontend targets different frequency ranges while modeling long-range dependencies and multi-level interactions by using two convolutional attention networks with attention-augmented convolution (AAC) blocks. The backend dynamically recalibrates multi-scale and level features extracted from the frontend by incorporating self-attention. The difference between MuSLCAT and MuSLCAN is their backend components. MuSLCAT's backend is a modified version of BERT. While MuSLCAN's is a simple AAC block. We validate the proposed MuSLCAT and MuSLCAN architectures by comparing them to state-of-the-art networks on four benchmark datasets for music tagging and genre recognition. Our experiments show that MuSLCAT and MuSLCAN consistently yield competitive results when compared to state-of-the-art waveform-based models yet require considerably fewer parameters.