Towards Effective Negation Modeling in Joint Audio-Text Models for Music

TL;DR

This paper enhances joint audio-text models for music retrieval by explicitly modeling negation through data augmentation and contrastive learning, improving their ability to distinguish between presence and absence of musical elements.

Contribution

It introduces a novel training approach with negation-aware data augmentation and contrastive loss, specifically targeting negation understanding in joint audio-text models.

Findings

Improved negation detection in retrieval and classification tasks.

Negation modeling enhances semantic understanding without harming retrieval performance.

Proposed protocols effectively evaluate negation handling.

Abstract

Joint audio-text models are widely used for music retrieval, yet they struggle with semantic phenomena such as negation. Negation is fundamental for distinguishing the absence (or presence) of musical elements (e.g., "with vocals" vs. "without vocals"), but current systems fail to represent this reliably. In this work, we investigate and mitigate this limitation by training CLAP models from scratch on the Million Song Dataset with LP-MusicCaps-MSD captions. We introduce negation through text augmentation and a dissimilarity-based contrastive loss, designed to explicitly separate original and negated captions in the joint embedding space. To evaluate progress, we propose two protocols that frame negation modeling as retrieval and binary classification tasks. Experiments demonstrate that both methods, individually and combined, improve negation handling while largely preserving retrieval performance.