Understanding the success and failure of online political debate: Experimental evidence using large language models

TL;DR

This study shows that adjusting elements like tone and willingness to compromise in online political debates can improve discussion quality but does not change people's political views.

Contribution

The paper experimentally identifies causal effects of justification, tone, and compromise on online debate quality using large language models.

Findings

Adjusting justification, tone, and willingness to compromise increases high-quality responses by 1.6 to 2 times.

Improved debate quality does not lead to changes in political attitudes.

Openness to alternative viewpoints increases with better debate approaches.

Abstract

Online political debate is frequently lamented for being toxic, partisan, and counterproductive. However, we know little about how core elements of political debate (justification, tone, willingness to compromise, and partisanship) affect its quality. Using text-based treatments experimentally manipulated with a large language model, we test how these elements causally affect the quality of open-text responses about issues important to the US and UK public. We find substantial evidence that differences in justification, tone, and willingness to compromise, but not partisanship, affect the quality of subsequent discourse. Combined, these elements increase the probability of high-quality responses by roughly 1.6 to 2 times and substantially increase openness to alternative viewpoints. Despite the ability to bring about substantial changes in discourse quality, we find no evidence of changes in political attitudes themselves. Our findings demonstrate how adapting approaches to online debate can foster healthy democratic interactions but have less influence on changing minds.

Adjusting tone, justification, and compromise improves online debate quality but does not change political attitudes.

Figures