Echoes in the Loop: Diagnosing Risks in LLM-Powered Recommender Systems under Feedback Loops

TL;DR

This paper introduces a diagnostic framework to analyze systemic risks like bias and hallucination in LLM-powered recommender systems, especially their propagation through feedback loops over time.

Contribution

It presents a role-aware, phase-wise diagnostic approach and a controlled feedback-loop pipeline for empirical risk measurement in LLM-based recommenders.

Findings

LLM components amplify popularity bias

Hallucinations introduce spurious signals

Feedback loops cause polarization and self-reinforcing exposure

Abstract

Large language models (LLMs) are increasingly embedded into recommender systems, where they operate across multiple functional roles such as data augmentation, profiling, and decision making. While prior work emphasizes recommendation performance, the systemic risks of LLMs, such as bias and hallucination, and their propagation through feedback loops remain largely unexplored. In this paper, we propose a role-aware, phase-wise diagnostic framework that traces how these risks emerge, manifest in ranking outcomes, and accumulate over repeated recommendation cycles. We formalize a controlled feedback-loop pipeline that simulates long-term interaction dynamics and enables empirical measurement of risks at the LLM-generated content, ranking, and ecosystem levels. Experiments on widely used benchmarks demonstrate that LLM-based components can amplify popularity bias, introduce spurious signals through hallucination, and lead to polarized and self-reinforcing exposure patterns over time. We plan to release our framework as an open-source toolkit to facilitate systematic risk analysis across diverse LLM-powered recommender systems.