News Source Citing Patterns in AI Search Systems

TL;DR

This study analyzes citation patterns in AI search systems, revealing biases towards certain news outlets, concentration among few sources, and minimal impact of source quality on user satisfaction, highlighting challenges in system design.

Contribution

It provides the first comprehensive analysis of news citation behaviors in AI search systems, uncovering biases, concentration patterns, and user preferences.

Findings

News citations are concentrated among few outlets.

Models exhibit a liberal bias in citing news sources.

User satisfaction is unaffected by source political leaning or quality.

Abstract

AI-powered search systems are emerging as new information gatekeepers, fundamentally transforming how users access news and information. Despite their growing influence, the citation patterns of these systems remain poorly understood. We address this gap by analyzing data from the AI Search Arena, a head-to-head evaluation platform for AI search systems. The dataset comprises over 24,000 conversations and 65,000 responses from models across three major providers: OpenAI, Perplexity, and Google. Among the over 366,000 citations embedded in these responses, 9% reference news sources. We find that while models from different providers cite distinct news sources, they exhibit shared patterns in citation behavior. News citations concentrate heavily among a small number of outlets and display a pronounced liberal bias, though low-credibility sources are rarely cited. User preference analysis reveals that neither the political leaning nor the quality of cited news sources significantly influences user satisfaction. These findings reveal significant challenges in current AI search systems and have important implications for their design and governance.