Investigating Memorization of Conspiracy Theories in Text Generation

TL;DR

This paper investigates how pretrained language models can generate conspiracy theories, revealing their tendency to memorize such content and highlighting the need for careful review of NLG models before deployment.

Contribution

It introduces a new dataset and methodology to test conspiracy theory memorization in language models, revealing their propensity to generate such content based on model size and temperature.

Findings

Many conspiracy theories are deeply rooted in pretrained models.

Model size and temperature influence conspiracy theory generation.

Highlights risks of memorization in language models.

Abstract

The adoption of natural language generation (NLG) models can leave individuals vulnerable to the generation of harmful information memorized by the models, such as conspiracy theories. While previous studies examine conspiracy theories in the context of social media, they have not evaluated their presence in the new space of generative language models. In this work, we investigate the capability of language models to generate conspiracy theory text. Specifically, we aim to answer: can we test pretrained generative language models for the memorization and elicitation of conspiracy theories without access to the model's training data? We highlight the difficulties of this task and discuss it in the context of memorization, generalization, and hallucination. Utilizing a new dataset consisting of conspiracy theory topics and machine-generated conspiracy theories helps us discover that many conspiracy theories are deeply rooted in the pretrained language models. Our experiments demonstrate a relationship between model parameters such as size and temperature and their propensity to generate conspiracy theory text. These results indicate the need for a more thorough review of NLG applications before release and an in-depth discussion of the drawbacks of memorization in generative language models.