Misinformation Exposure in the Chinese Web: A Cross-System Evaluation of Search Engines, LLMs, and AI Overviews

TL;DR

This study evaluates the factual accuracy of search engines, LLMs, and AI overviews in the Chinese web ecosystem using a new dataset, revealing significant variability and potential misinformation exposure among users.

Contribution

Introduces a large Chinese fact-checking dataset and a unified evaluation pipeline to compare information access systems, highlighting differences in accuracy and regional misinformation risks.

Findings

Substantial accuracy differences across systems

Topic-level variability in factual correctness

Estimated regional misinformation exposure among Chinese users

Abstract

Large Language Models (LLMs) are increasingly integrated into search services, providing direct answers that can reduce users' reliance on traditional result pages. Yet their factual reliability in non-English web ecosystems remains poorly understood, particularly when answering real user queries. We introduce a fact-checking dataset of 12~161 Chinese Yes/No questions derived from real-world online search logs and develop a unified evaluation pipeline to compare three information-access paradigms: traditional search engines, standalone LLMs, and AI-generated overview modules. Our analysis reveals substantial differences in factual accuracy and topic-level variability across systems. By combining this performance with real-world Baidu Index statistics, we further estimate potential exposure to incorrect factual information of Chinese users across regions. These findings highlight structural risks in AI-mediated search and underscore the need for more reliable and transparent information-access tools for the digital world.