Sound Check: Auditing Audio Datasets

TL;DR

This paper audits prominent audio datasets for biases, toxicity, and copyright issues, revealing significant ethical concerns and providing a web tool for dataset exploration to promote transparency and accountability.

Contribution

It is the first comprehensive audit of audio datasets addressing bias, toxicity, and copyright, and introduces an interactive web tool for dataset analysis.

Findings

Datasets are biased against women.

Presence of toxic stereotypes about marginalized groups.

Significant copyrighted material detected.

Abstract

Generative audio models are rapidly advancing in both capabilities and public utilization -- several powerful generative audio models have readily available open weights, and some tech companies have released high quality generative audio products. Yet, while prior work has enumerated many ethical issues stemming from the data on which generative visual and textual models have been trained, we have little understanding of similar issues with generative audio datasets, including those related to bias, toxicity, and intellectual property. To bridge this gap, we conducted a literature review of hundreds of audio datasets and selected seven of the most prominent to audit in more detail. We found that these datasets are biased against women, contain toxic stereotypes about marginalized communities, and contain significant amounts of copyrighted work. To enable artists to see if they are in popular audio datasets and facilitate exploration of the contents of these datasets, we developed a web tool audio datasets exploration tool at https://audio-audit.vercel.app.