Guiding without Generating: Artificial Intelligence (AI)-Enabled Topic Nudges in Online Reviews

TL;DR

This study examines how AI-enabled topic nudges in online reviews influence review content, length, complexity, and helpfulness, revealing both benefits in coverage and drawbacks in readability and perceived usefulness.

Contribution

It provides empirical evidence on AI-guided prompts shaping user-generated content, highlighting effects on review diversity, length, complexity, and helpfulness, with insights into user heterogeneity.

Findings

Nudges increase topical coverage and review length.

Reviews become more complex and less readable.

Helpfulness votes decrease, especially when content lacks focus.

Abstract

Digital platforms increasingly face a common challenge in the age of artificial intelligence (AI): how to elicit richer and more useful user-generated content (UGC) without fully automating content production. We study this question in the context of online reviews by examining Yelp's introduction of an AI-enabled topic nudging tool in 2023, which provides real-time prompts to guide reviewers in addressing key dimensions of the dining experience as they write. Using more than 1.5 million Yelp reviews and a differences-in-differences design, we find that AI-enabled topic nudges significantly reshape review generation. The nudges expand topical coverage, especially for underrepresented aspects such as service and ambiance, and lead to longer reviews, but they also reduce overall review volume. In addition, reviews become more textually complex and less readable, and receive fewer helpfulness votes on average. Further analysis shows that the decline in perceived helpfulness is mitigated when review content remains concentrated on a dominant dimension, highlighting the importance of informational focus. We also find heterogeneous effects: less experienced users expand topical coverage and review length more strongly, whereas experienced users exhibit greater complexity and larger declines in perceived helpfulness. Our findings extend research on AI and UGC by highlighting a distinct mode of AI deployment-guiding human contributions rather than generating content on users' behalf-and by revealing its benefits and unintended consequences for platform design.