Counterspeech for Mitigating the Influence of Media Bias: Comparing Human and LLM-Generated Responses

TL;DR

This paper investigates counterspeech as a method to combat media bias and harmful comments, introducing a new dataset and comparing human and AI-generated responses to improve counterspeech effectiveness.

Contribution

It is the first study to explore counterspeech generation for news bias, providing a new dataset and methods to enhance AI-generated counterspeech.

Findings

Over 70% offensive comments support biased articles

Model-generated counterspeech is more polite but less diverse

Few-shot learning and background info improve counterspeech quality

Abstract

Biased news contributes to societal polarization and is often reinforced by hostile reader comments, constituting a vital yet often overlooked aspect of news dissemination. Our study reveals that offensive comments support biased content, amplifying bias and causing harm to targeted groups or individuals. Counterspeech is an effective approach to counter such harmful speech without violating freedom of speech, helping to limit the spread of bias. To the best of our knowledge, this is the first study to explore counterspeech generation in the context of news articles. We introduce a manually annotated dataset linking media bias, offensive comments, and counterspeech. We conduct a detailed analysis showing that over 70\% offensive comments support biased articles, amplifying bias and thus highlighting the importance of counterspeech generation. Comparing counterspeech generated by humans and large language models, we find model-generated responses are more polite but lack the novelty and diversity. Finally, we improve generated counterspeech through few-shot learning and integration of news background information, enhancing both diversity and relevance.