Taming Toxic Talk: Using chatbots to intervene with users posting toxic comments

TL;DR

This study investigates whether AI chatbots can rehabilitate users who post toxic comments online, finding that while users engage positively, there is no significant reduction in toxic behavior over a month.

Contribution

First large-scale field experiment assessing AI chatbot intervention effectiveness for online toxicity, providing insights into rehabilitative approaches.

Findings

Participants engaged in good faith conversations.

Many expressed remorse or desire to change.

No significant reduction in toxic behavior observed.

Abstract

Generative AI chatbots have proven surprisingly effective at persuading people to change their beliefs and attitudes in lab settings. However, the practical implications of these findings are not yet clear. In this work, we explore the impact of rehabilitative conversations with generative AI chatbots on users who share toxic content online. Toxic behaviors -- like insults or threats of violence, are widespread in online communities. Strategies to deal with toxic behavior are typically punitive, such as removing content or banning users. Rehabilitative approaches are rarely attempted, in part due to the emotional and psychological cost of engaging with aggressive users. In collaboration with seven large Reddit communities, we conducted a large-scale field experiment (N=893) to invite people who had recently posted toxic content to participate in conversations with AI chatbots. A qualitative analysis of the conversations shows that many participants engaged in good faith and even expressed remorse or a desire to change. However, we did not observe a significant change in toxic behavior in the following month compared to a control group. We discuss possible explanations for our findings, as well as theoretical and practical implications based on our results.