What's New in My Data? Novelty Exploration via Contrastive Generation

TL;DR

This paper introduces Contrastive Generative Exploration (CGE), a method for identifying novel properties in fine-tuning datasets by contrasting pre-trained and fine-tuned models, with an iterative approach to enhance diversity and effectiveness.

Contribution

The paper proposes a novel contrastive generation method for novelty discovery in datasets without direct data access, including an iterative version to improve diversity of generated examples.

Findings

CGE effectively detects novel content like toxic language and new languages.

The iterative CGE enhances diversity and captures a broader range of dataset properties.

CGE remains effective even with privacy-preserving fine-tuning methods.

Abstract

Fine-tuning is widely used to adapt language models for specific goals, often leveraging real-world data such as patient records, customer-service interactions, or web content in languages not covered in pre-training. These datasets are typically massive, noisy, and often confidential, making their direct inspection challenging. However, understanding them is essential for guiding model deployment and informing decisions about data cleaning or suppressing any harmful behaviors learned during fine-tuning. In this study, we introduce the task of novelty discovery through generation, which aims to identify novel properties of a fine-tuning dataset by generating examples that illustrate these properties. Our approach, Contrastive Generative Exploration (CGE), assumes no direct access to the data but instead relies on a pre-trained model and the same model after fine-tuning. By contrasting the predictions of these two models, CGE can generate examples that highlight novel characteristics of the fine-tuning data. However, this simple approach may produce examples that are too similar to one another, failing to capture the full range of novel phenomena present in the dataset. We address this by introducing an iterative version of CGE, where the previously generated examples are used to update the pre-trained model, and this updated model is then contrasted with the fully fine-tuned model to generate the next example, promoting diversity in the generated outputs. Our experiments demonstrate the effectiveness of CGE in detecting novel content, such as toxic language, as well as new natural and programming languages. Furthermore, we show that CGE remains effective even when models are fine-tuned using differential privacy techniques.