Membership and Dataset Inference Attacks on Large Audio Generative Models

TL;DR

This paper explores the potential for verifying if specific audio works were used in training large generative models, finding dataset inference to be a promising tool for copyright protection despite limited success of individual membership inference.

Contribution

It demonstrates that dataset inference can effectively determine if an artist's collection was included in training data for large audio models, advancing copyright protection methods.

Findings

Membership inference is limited at scale due to weak signals.

Dataset inference successfully aggregates evidence across multiple samples.

Dataset inference offers a practical approach for copyright accountability.

Abstract

Generative audio models, based on diffusion and autoregressive architectures, have advanced rapidly in both quality and expressiveness. This progress, however, raises pressing copyright concerns, as such models are often trained on vast corpora of artistic and commercial works. A central question is whether one can reliably verify if an artist's material was included in training, thereby providing a means for copyright holders to protect their content. In this work, we investigate the feasibility of such verification through membership inference attacks (MIA) on open-source generative audio models, which attempt to determine whether a specific audio sample was part of the training set. Our empirical results show that membership inference alone is of limited effectiveness at scale, as the per-sample membership signal is weak for models trained on large and diverse datasets. However, artists and media owners typically hold collections of works rather than isolated samples. Building on prior work in text and vision domains, in this work we focus on dataset inference (DI), which aggregates diverse membership evidence across multiple samples. We find that DI is successful in the audio domain, offering a more practical mechanism for assessing whether an artist's works contributed to model training. Our results suggest DI as a promising direction for copyright protection and dataset accountability in the era of large audio generative models.