The Empirical Impact of Data Sanitization on Language Models

TL;DR

This paper empirically examines how data sanitization, especially redacting sensitive information, affects language model performance across various NLP tasks, revealing task-dependent impacts and proposing mitigation strategies.

Contribution

It provides a comprehensive analysis of data sanitization effects on language models, highlighting task-specific impacts and introducing methods to mitigate performance degradation.

Findings

Low impact (1-5%) on sentiment analysis and entailment tasks.

Significant performance drop (>25%) on comprehension Q&A tasks.

Proposed content-based subsampling to repair redacted datasets.

Abstract

Data sanitization in the context of language modeling involves identifying sensitive content, such as personally identifiable information (PII), and redacting them from a dataset corpus. It is a common practice used in natural language processing (NLP) to maintain privacy. Nevertheless, the impact of data sanitization on the language understanding capability of a language model remains less studied. This paper empirically analyzes the effects of data sanitization across several benchmark language-modeling tasks including comprehension question answering (Q&A), entailment, sentiment analysis, and text classification. Our experiments cover a wide spectrum comprising finetuning small-scale language models, to prompting large language models (LLMs), on both original and sanitized datasets, and comparing their performance across the tasks. Interestingly, our results suggest that for some tasks such as sentiment analysis or entailment, the impact of redaction is quite low, typically around 1-5%, while for tasks such as comprehension Q&A there is a big drop of >25% in performance observed in redacted queries as compared to the original. For tasks that have a higher impact, we perform a deeper dive to inspect the presence of task-critical entities. Finally, we investigate correlation between performance and number of redacted entities, and also suggest a strategy to repair an already redacted dataset by means of content-based subsampling. Additional details are available at https://sites.google.com/view/datasan.