Detecting Bot-Generated Text by Characterizing Linguistic Accommodation in Human-Bot Interactions

TL;DR

This paper investigates how linguistic accommodation in human-bot interactions can be used to detect bot-generated text more effectively across datasets and models, offering insights into differences in communication patterns.

Contribution

It introduces a novel detection approach leveraging linguistic accommodation, enhancing robustness over existing methods by focusing on human responses rather than bot text.

Findings

Detection methods based on human responses are more robust across datasets.

Linguistic alignment differs significantly between human-human and human-bot conversations.

Analyzing linguistic accommodation provides valuable insights into interaction dynamics.

Abstract

Language generation models' democratization benefits many domains, from answering health-related questions to enhancing education by providing AI-driven tutoring services. However, language generation models' democratization also makes it easier to generate human-like text at-scale for nefarious activities, from spreading misinformation to targeting specific groups with hate speech. Thus, it is essential to understand how people interact with bots and develop methods to detect bot-generated text. This paper shows that bot-generated text detection methods are more robust across datasets and models if we use information about how people respond to it rather than using the bot's text directly. We also analyze linguistic alignment, providing insight into differences between human-human and human-bot conversations.