LLM-POTUS Score: A Framework of Analyzing Presidential Debates with Large Language Models

TL;DR

This paper presents a new framework using large language models to objectively analyze presidential debates by assessing candidate strategies and audience impact, providing nuanced, multi-dimensional performance scores.

Contribution

It introduces the LLM-POTUS Score framework that evaluates debate performance based on candidate policies and audience interests, advancing political discourse analysis with LLMs.

Findings

The framework effectively analyzes debate transcripts for nuanced performance insights.

It reveals how debate strategies influence different audience groups.

The method offers an independent tool for citizens to evaluate debates.

Abstract

Large language models have demonstrated remarkable capabilities in natural language processing, yet their application to political discourse analysis remains underexplored. This paper introduces a novel approach to evaluating presidential debate performances using LLMs, addressing the longstanding challenge of objectively assessing debate outcomes. We propose a framework that analyzes candidates' "Policies, Persona, and Perspective" (3P) and how they resonate with the "Interests, Ideologies, and Identity" (3I) of four key audience groups: voters, businesses, donors, and politicians. Our method employs large language models to generate the LLM-POTUS Score, a quantitative measure of debate performance based on the alignment between 3P and 3I. We apply this framework to analyze transcripts from recent U.S. presidential debates, demonstrating its ability to provide nuanced, multi-dimensional assessments of candidate performances. Our results reveal insights into the effectiveness of different debating strategies and their impact on various audience segments. This study not only offers a new tool for political analysis but also explores the potential and limitations of using LLMs as impartial judges in complex social contexts. In addition, this framework provides individual citizens with an independent tool to evaluate presidential debate performances, which enhances democratic engagement and reduces reliance on potentially biased media interpretations and institutional influence, thereby strengthening the foundation of informed civic participation.