An Item Response Theory Framework for Persuasion

TL;DR

This paper introduces an Item Response Theory framework to analyze argument persuasiveness in language, demonstrating its effectiveness across multiple datasets and highlighting the importance of style and content representations.

Contribution

It applies IRT to language persuasion analysis, providing a novel approach and empirical evaluation on new and existing datasets.

Findings

IRT effectively models argument persuasiveness

Speaker embeddings correlate with real-world persuadability

Model outperforms baseline approaches

Abstract

In this paper, we apply Item Response Theory, popular in education and political science research, to the analysis of argument persuasiveness in language. We empirically evaluate the model's performance on three datasets, including a novel dataset in the area of political advocacy. We show the advantages of separating these components under several style and content representations, including evaluating the ability of the speaker embeddings generated by the model to parallel real-world observations about persuadability.