A Common Pool of Privacy Problems: Legal and Technical Lessons from a Large-Scale Web-Scraped Machine Learning Dataset

TL;DR

This paper examines the privacy risks of large-scale web-scraped datasets for AI, revealing significant personally identifiable information and legal concerns despite sanitization efforts.

Contribution

It provides an empirical analysis of a popular dataset showing privacy issues and discusses legal implications, advocating for stricter data curation standards.

Findings

Significant presence of personally identifiable information in the dataset

Legal analysis highlights risks under current privacy laws

Current scraping practices may propagate personal data into AI models

Abstract

We investigate the contents of web-scraped data for training AI systems, at sizes where human dataset curators and compilers no longer manually annotate every sample. Building off of prior privacy concerns in machine learning models, we ask: What are the legal privacy implications of web-scraped machine learning datasets? In an empirical study of a popular training dataset, we find significant presence of personally identifiable information despite sanitization efforts. Our audit provides concrete evidence to support the concern that any large-scale web-scraped dataset may contain legally defined personal data. We use these findings of a real-world dataset to inform our legal analysis with respect to existing privacy and data protection laws. We surface various legal risks of current data curation practices that may propagate personal information to train downstream models. Based on our empirical and legal analyses, we argue for reorientation of current frameworks of "publicly available" information to meaningfully limit the development of AI built upon indiscriminate scraping of the internet.