DAIRHuM: A Platform for Directly Aligning AI Representations with Human Musical Judgments applied to Carnatic Music

TL;DR

This paper introduces DAIRHuM, a platform for aligning AI music representations with human judgments, applied to Carnatic music, addressing data scarcity and cultural specificity in MIR research.

Contribution

It presents a novel platform enabling exploration of human-AI alignment in music, specifically tailored for under-represented genres like Carnatic music.

Findings

Significant alignment between NSynth representations and human rhythmic judgments.

Key differences identified in rhythm perception specific to Carnatic music.

Platform facilitates analysis despite data scarcity.

Abstract

Quantifying and aligning music AI model representations with human behavior is an important challenge in the field of MIR. This paper presents a platform for exploring the Direct alignment between AI music model Representations and Human Musical judgments (DAIRHuM). It is designed to enable musicians and experimentalists to label similarities in a dataset of music recordings, and examine a pre-trained model's alignment with their labels using quantitative scores and visual plots. DAIRHuM is applied to analyze alignment between NSynth representations, and a rhythmic duet between two percussionists in a Carnatic quartet ensemble, an example of a genre where annotated data is scarce and assessing alignment is non-trivial. The results demonstrate significant findings on model alignment with human judgments of rhythmic harmony, while highlighting key differences in rhythm perception and music similarity judgments specific to Carnatic music. This work is among the first efforts to enable users to explore human-AI model alignment in Carnatic music and advance MIR research in Indian music while dealing with data scarcity and cultural specificity. The development of this platform provides greater accessibility to music AI tools for under-represented genres.