Headlines You Won't Forget: Can Pronoun Insertion Increase Memorability?

TL;DR

This study investigates whether inserting pronouns into news headlines can enhance memorability, using experiments and language models, but results are mixed and context-dependent.

Contribution

It explores the effect of pronoun insertion on memorability and evaluates the feasibility and quality of automatic revisions by large language models.

Findings

Pronoun insertion has mixed effects on headline memorability.

Effects vary based on topic, insertion method, and context.

Automatic revisions by LLMs often lack accuracy and naturalness.

Abstract

For news headlines to influence beliefs and drive action, relevant information needs to be retained and retrievable from memory. In this probing study we draw on experiment designs from cognitive psychology to examine how a specific linguistic feature, namely direct address through first- and second-person pronouns, affects memorability and to what extent it is feasible to use large language models for the targeted insertion of such a feature into existing text without changing its core meaning. Across three controlled memorization experiments with a total of 240 participants, yielding 7,680 unique memory judgments, we show that pronoun insertion has mixed effects on memorability. Exploratory analyses indicate that effects differ based on headline topic, how pronouns are inserted and their immediate contexts. Additional data and fine-grained analysis is needed to draw definitive conclusions on these mediating factors. We further show that automatic revisions by LLMs are not always appropriate: Crowdsourced evaluations find many of them to be lacking in content accuracy and emotion retention or resulting in unnatural writing style. We make our collected data available for future work.