Prosocial Persuasion at Scale? Large Language Models Outperform Humans in Donation Appeals Across Levels of Personalization

TL;DR

Large Language Models can generate donation appeals that outperform human-written content in persuading prosocial behavior, especially with proper personalization, as shown in two large online experiments.

Contribution

This study demonstrates that LLM-generated donation appeals are more effective than human ones across various levels of personalization, highlighting their potential for prosocial persuasion.

Findings

LLM-generated appeals lead to more donations than human-authored ones.

Personalization increases the effectiveness of donation appeals.

False personalization decreases persuasive impact.

Abstract

Large Language Models (LLMs) are increasingly regarded as having the potential to generate persuasive content at scale. While previous studies have focused on the risks associated with LLM-generated misinformation, the role of LLMs in enabling prosocial persuasion is still underexplored. We investigate whether donation appeals authored by LLMs are as effective as those written by humans across degrees of personalization. Two preregistered online experiments (Study 1: N = 658; Study 2: N = 642) manipulated Personalization (generic vs. personalized vs. falsely personalized) and Content source (human vs. LLM) and presented participants with donation appeals for charities. We assessed how participants distributed their bonus money across the charities, how they engaged with the donation appeals, and how persuasive they found them. In both experiments, LLM-generated content yielded more donations, resulted in higher engagement, and was rated as more persuasive than human-authored content. There was a gain associated with personalization (Study 2) and a penalty for false personalization (Study 1). Our results suggest that LLMs may be a suitable technology for generating content that can encourage prosocial behavior.