Learning Unsupervised Hierarchies of Audio Concepts

TL;DR

This paper introduces a method to learn and hierarchically organize high-level music concepts from audio data, addressing the complexity of non-independent, mixed nature of music attributes, and demonstrating alignment with ground-truth and proxy concept hierarchies.

Contribution

It adapts concept learning to music, enabling automatic hierarchy discovery of interconnected music concepts from audio signals.

Findings

Hierarchies align with ground-truth when available

Hierarchies correspond with proxy concept similarities

Method effectively captures complex music concept relationships

Abstract

Music signals are difficult to interpret from their low-level features, perhaps even more than images: e.g. highlighting part of a spectrogram or an image is often insufficient to convey high-level ideas that are genuinely relevant to humans. In computer vision, concept learning was therein proposed to adjust explanations to the right abstraction level (e.g. detect clinical concepts from radiographs). These methods have yet to be used for MIR. In this paper, we adapt concept learning to the realm of music, with its particularities. For instance, music concepts are typically non-independent and of mixed nature (e.g. genre, instruments, mood), unlike previous work that assumed disentangled concepts. We propose a method to learn numerous music concepts from audio and then automatically hierarchise them to expose their mutual relationships. We conduct experiments on datasets of playlists from a music streaming service, serving as a few annotated examples for diverse concepts. Evaluations show that the mined hierarchies are aligned with both ground-truth hierarchies of concepts -- when available -- and with proxy sources of concept similarity in the general case.