Exploring Musical Roots: Applying Audio Embeddings to Empower Influence Attribution for a Generative Music Model

TL;DR

This paper introduces a methodology using audio embeddings to identify and attribute musical influences in generative models, enhancing transparency and understanding of training data influence.

Contribution

It presents a systematic approach employing CLMR and CLAP embeddings for music similarity measurement, validated through human studies and analysis of audio modifications.

Findings

CLMR and CLAP embeddings effectively measure music similarity.

Audio modifications impact similarity scores, affecting influence attribution.

The methodology enables automated influence attribution in generative music models.

Abstract

Every artist has a creative process that draws inspiration from previous artists and their works. Today, "inspiration" has been automated by generative music models. The black box nature of these models obscures the identity of the works that influence their creative output. As a result, users may inadvertently appropriate, misuse, or copy existing artists' works. We establish a replicable methodology to systematically identify similar pieces of music audio in a manner that is useful for understanding training data attribution. A key aspect of our approach is to harness an effective music audio similarity measure. We compare the effect of applying CLMR and CLAP embeddings to similarity measurement in a set of 5 million audio clips used to train VampNet, a recent open source generative music model. We validate this approach with a human listening study. We also explore the effect that modifications of an audio example (e.g., pitch shifting, time stretching, background noise) have on similarity measurements. This work is foundational to incorporating automated influence attribution into generative modeling, which promises to let model creators and users move from ignorant appropriation to informed creation. Audio samples that accompany this paper are available at https://tinyurl.com/exploring-musical-roots.