Diagnosing and Debiasing Corpus-Based Political Bias and Insults in GPT2

TL;DR

This paper explores methods for detecting and reducing political bias and insults in GPT-2 models, enhancing their ethical and social responsibility by improving self-diagnosis and self-debiasing techniques.

Contribution

It extends existing self-diagnosis and self-debiasing methods to effectively mitigate political bias and insults in GPT-2, addressing gaps in bias types handled by prior work.

Findings

Self-diagnosis can identify biases in generated content.

Self-debiasing reduces the likelihood of harmful outputs.

Effective mitigation of political bias and insults in GPT-2.

Abstract

The training of large language models (LLMs) on extensive, unfiltered corpora sourced from the internet is a common and advantageous practice. Consequently, LLMs have learned and inadvertently reproduced various types of biases, including violent, offensive, and toxic language. However, recent research shows that generative pretrained transformer (GPT) language models can recognize their own biases and detect toxicity in generated content, a process referred to as self-diagnosis. In response, researchers have developed a decoding algorithm that allows LLMs to self-debias, or reduce their likelihood of generating harmful text. This study investigates the efficacy of the diagnosing-debiasing approach in mitigating two additional types of biases: insults and political bias. These biases are often used interchangeably in discourse, despite exhibiting potentially dissimilar semantic and syntactic properties. We aim to contribute to the ongoing effort of investigating the ethical and social implications of human-AI interaction.