JamendoMaxCaps: A Large Scale Music-caption Dataset with Imputed Metadata

TL;DR

JamendoMaxCaps is a large-scale, publicly available music-caption dataset that combines machine-generated captions with imputed metadata to enhance research in music-language understanding and retrieval.

Contribution

The paper introduces JamendoMaxCaps, a novel large-scale music dataset with generated captions and metadata imputation using a retrieval system and large language models.

Findings

Validated the effectiveness of metadata imputation with five measurements

Demonstrated improved music retrieval and understanding tasks

Provided a new resource for multimodal music research

Abstract

We introduce JamendoMaxCaps, a large-scale music-caption dataset featuring over 362,000 freely licensed instrumental tracks from the renowned Jamendo platform. The dataset includes captions generated by a state-of-the-art captioning model, enhanced with imputed metadata. We also introduce a retrieval system that leverages both musical features and metadata to identify similar songs, which are then used to fill in missing metadata using a local large language model (LLLM). This approach allows us to provide a more comprehensive and informative dataset for researchers working on music-language understanding tasks. We validate this approach quantitatively with five different measurements. By making the JamendoMaxCaps dataset publicly available, we provide a high-quality resource to advance research in music-language understanding tasks such as music retrieval, multimodal representation learning, and generative music models.