Be like a Goldfish, Don't Memorize! Mitigating Memorization in Generative LLMs

TL;DR

This paper introduces the goldfish loss, a training modification for large language models that reduces memorization of training data by excluding random token subsets during training, thereby enhancing privacy without harming performance.

Contribution

It proposes a novel training objective, the goldfish loss, which effectively mitigates memorization in large language models while maintaining their utility.

Findings

Significant reduction in memorization of training data.

Minimal impact on downstream task performance.

Effective across billion-scale Llama-2 models.

Abstract

Large language models can memorize and repeat their training data, causing privacy and copyright risks. To mitigate memorization, we introduce a subtle modification to the next-token training objective that we call the goldfish loss. During training, randomly sampled subsets of tokens are excluded from the loss computation. These dropped tokens are not memorized by the model, which prevents verbatim reproduction of a complete chain of tokens from the training set. We run extensive experiments training billion-scale Llama-2 models, both pre-trained and trained from scratch, and demonstrate significant reductions in extractable memorization with little to no impact on downstream benchmarks.