Balancing Domestic and Global Perspectives: Evaluating Dual-Calibration and LLM-Generated Nudges for Diverse News Recommendation

TL;DR

This paper evaluates dual-calibration algorithms and LLM-based nudges to promote diverse news consumption, showing that algorithmic nudges effectively increase diversity and that personalized presentation influences user engagement.

Contribution

Introduces a novel topic-locality dual calibration algorithm and LLM-based news presentation nudges, tested through a real-user study on news diversity.

Findings

Algorithmic nudges increase news diversity exposure.

Personalized relevance highlighting improves user engagement.

Long-term exposure shifts reading habits towards balanced news.

Abstract

In this study, we applied the ``personalized diversity nudge framework'' with the goal of expanding user reading coverage in terms of news locality (i.e., domestic and world news). We designed a novel topic-locality dual calibration algorithmic nudge and a large language model-based news personalization presentation nudge, then launched a 5-week real-user study with 120 U.S. news readers on the news recommendation experiment platform POPROX. With user interaction logs and survey responses, we found that algorithmic nudges can successfully increase exposure and consumption diversity, while the impact of LLM-based presentation nudges varied. User-level topic interest is a strong predictor of user clicks, while highlighting the relevance of news articles to prior read articles outperforms generic topic-based and no personalization. We also demonstrate that longitudinal exposure to calibrated news may shift readers' reading habits to value a balanced news digest from both domestic and world articles. Our results provide direction for future work on nudging for diverse consumption in news recommendation systems.