How Much Does Persuasion Strategy Matter? LLM-Annotated Evidence from Charitable Donation Dialogues

TL;DR

This study examines the impact of persuasion strategies in charitable donation dialogues, finding that most strategies have little effect, with guilt induction reducing donations and reciprocity being beneficial.

Contribution

It provides a comprehensive annotation of persuasion strategies in a large dialogue corpus using multiple LLMs, revealing limited variance explained by strategies and highlighting guilt's counterproductive role.

Findings

Strategy categories explain little variance in donation outcomes.

Guilt induction significantly decreases donation rates.

Reciprocity is a consistent positive predictor of donations.

Abstract

Which persuasion strategies, if any, are associated with donation compliance? Answering this requires fine-grained strategy labels across a full corpus and statistical tests corrected for multiple comparisons. We annotate all 10,600 persuader turns in the 1,017-dialogue PersuasionForGood corpus (Wang et al., 2019), where donation outcomes are directly observable, with a taxonomy of 41 strategies in 11 categories, using three open-source large language models (LLMs; Qwen3:30b, Mistral-Small-3.2, Phi-4). Strategy categories alone explain little variance in donation outcome (pseudo $R^2 \approx 0.015$, consistent across all three annotators). Guilt Induction is the only strategy significantly associated with lower donation rates ($\Delta \approx -23$ percentage points), an effect that replicates across all three models despite only moderate inter-model agreement. Reciprocity is the most robust positive correlate. Target sentiment and interest predict whether a donation occurs but show at most a weak correlation with donation amount. These findings suggest that strategy identification alone is insufficient to explain persuasion effectiveness, and that guilt-based appeals may be counterproductive in prosocial settings. We release the fully annotated corpus as a public resource.