BeatFM: Improving Beat Tracking with Pre-trained Music Foundation Model

TL;DR

BeatFM leverages a pre-trained music foundation model with a multi-dimensional semantic aggregation module to significantly improve beat and downbeat tracking accuracy across diverse datasets.

Contribution

The paper introduces BeatFM, a novel beat tracking approach that utilizes a pre-trained music foundation model and a multi-dimensional semantic aggregation module for enhanced performance.

Findings

Achieves state-of-the-art beat and downbeat tracking results.

Effectively generalizes across diverse musical styles.

Demonstrates robustness on multiple benchmark datasets.

Abstract

Beat tracking is a widely researched topic in music information retrieval. However, current beat tracking methods face challenges due to the scarcity of labeled data, which limits their ability to generalize across diverse musical styles and accurately capture complex rhythmic structures. To overcome these challenges, we propose a novel beat tracking paradigm BeatFM, which introduces a pre-trained music foundation model and leverages its rich semantic knowledge to improve beat tracking performance. Pre-training on diverse music datasets endows music foundation models with a robust understanding of music, thereby effectively addressing these challenges. To further adapt it for beat tracking, we design a plug-and-play multi-dimensional semantic aggregation module, which is composed of three parallel sub-modules, each focusing on semantic aggregation in the temporal, frequency, and channel domains, respectively. Extensive experiments demonstrate that our method achieves state-of-the-art performance in beat and downbeat tracking across multiple benchmark datasets.