LLM-Supported Content Analysis of Motivated Reasoning on Climate Change

TL;DR

This paper uses large language models to analyze YouTube comments on climate change, revealing motivated reasoning patterns and ideological divides in discourse, with implications for understanding public polarization.

Contribution

It introduces a novel LLM-based method for large-scale qualitative analysis of online climate change discussions, combining topic annotation with social network analysis.

Findings

Comments on policy and natural cycles have lower engagement.

Misinformation attracts more interaction.

Discourse reflects motivated reasoning and ideological biases.

Abstract

Public discourse around climate change remains polarized despite scientific consensus on anthropogenic climate change (ACC). This study examines how "believers" and "skeptics" of ACC differ in their YouTube comment discourse. We analyzed 44,989 comments from 30 videos using a large language model (LLM) as a qualitative annotator, identifying ten distinct topics. These annotations were combined with social network analysis to examine engagement patterns. A linear mixed-effects model showed that comments about government policy and natural cycles generated significantly lower interaction compared to misinformation, suggesting these topics are ideologically settled points within communities. These patterns reflect motivated reasoning, where users selectively engage with content that aligns with their identity and beliefs. Our findings highlight the utility of LLMs for large-scale qualitative analysis and highlight how climate discourse is shaped not only by content, but by underlying cognitive and ideological motivations.