ELF22: A Context-based Counter Trolling Dataset to Combat Internet Trolls

TL;DR

This paper introduces ELF22, a new dataset designed to generate context-aware counter responses to online trolls, aiming to reduce social harm and promote healthier online discussions.

Contribution

It presents a novel pair-wise dataset with labeled response strategies for training models to generate effective counter responses to trolls.

Findings

Models fine-tuned on ELF22 outperform baselines in strategy-controlled response generation.

Human evaluation confirms improved response quality and strategy adherence.

The dataset enables targeted counter-trolling to mitigate online trolling effects.

Abstract

Online trolls increase social costs and cause psychological damage to individuals. With the proliferation of automated accounts making use of bots for trolling, it is difficult for targeted individual users to handle the situation both quantitatively and qualitatively. To address this issue, we focus on automating the method to counter trolls, as counter responses to combat trolls encourage community users to maintain ongoing discussion without compromising freedom of expression. For this purpose, we propose a novel dataset for automatic counter response generation. In particular, we constructed a pair-wise dataset that includes troll comments and counter responses with labeled response strategies, which enables models fine-tuned on our dataset to generate responses by varying counter responses according to the specified strategy. We conducted three tasks to assess the effectiveness of our dataset and evaluated the results through both automatic and human evaluation. In human evaluation, we demonstrate that the model fine-tuned on our dataset shows a significantly improved performance in strategy-controlled sentence generation.