Universal Music Representations? Evaluating Foundation Models on World Music Corpora

TL;DR

This paper evaluates how well current foundation models generalize across diverse musical traditions, revealing their strengths and limitations in capturing world music representations.

Contribution

It provides a comprehensive evaluation of foundation models on multiple world music corpora, introducing methodologies and benchmarks for assessing cross-cultural music understanding.

Findings

Larger models generally perform better on non-Western music.

Model performance declines with increasing cultural distance.

The proposed evaluation framework sets new benchmarks for future research.

Abstract

Foundation models have revolutionized music information retrieval, but questions remain about their ability to generalize across diverse musical traditions. This paper presents a comprehensive evaluation of five state-of-the-art audio foundation models across six musical corpora spanning Western popular, Greek, Turkish, and Indian classical traditions. We employ three complementary methodologies to investigate these models' cross-cultural capabilities: probing to assess inherent representations, targeted supervised fine-tuning of 1-2 layers, and multi-label few-shot learning for low-resource scenarios. Our analysis shows varying cross-cultural generalization, with larger models typically outperforming on non-Western music, though results decline for culturally distant traditions. Notably, our approaches achieve state-of-the-art performance on five out of six evaluated datasets, demonstrating the effectiveness of foundation models for world music understanding. We also find that our targeted fine-tuning approach does not consistently outperform probing across all settings, suggesting foundation models already encode substantial musical knowledge. Our evaluation framework and benchmarking results contribute to understanding how far current models are from achieving universal music representations while establishing metrics for future progress.