OMAR-RQ: Open Music Audio Representation Model Trained with Multi-Feature Masked Token Prediction

TL;DR

OMAR-RQ is a self-supervised open-source music audio model trained on 330,000 hours of data, achieving state-of-the-art results in various music understanding tasks and promoting accessible research tools.

Contribution

The paper introduces OMAR-RQ, a novel self-supervised music audio model trained on large-scale data, with extensive experiments on input features and quantization, setting new performance benchmarks.

Findings

Achieved state-of-the-art performance in music tagging and pitch estimation.

Demonstrated effectiveness across multiple music understanding tasks.

Provided open-source code and models for community use.

Abstract

Developing open-source foundation models is essential for advancing research in music audio understanding and ensuring access to powerful, multipurpose representations for music information retrieval. We present OMAR-RQ, a model trained with self-supervision via masked token classification methodologies using a large-scale dataset with over 330,000 hours of music audio. We experiment with different input features and quantization options, and achieve state-of-the-art performance in music tagging, pitch estimation, chord recognition, beat tracking, segmentation, and difficulty estimation among open self-supervised models. We open-source our training and evaluation pipelines and model weights, available at https://github.com/mtg/omar-rq.