Search Engines in an AI Era: The False Promise of Factual and Verifiable Source-Cited Responses

TL;DR

This paper critically examines the limitations of LLM-based answer engines, revealing common issues like hallucinations and citation inaccuracies, and introduces an evaluation benchmark to improve their transparency and reliability.

Contribution

It provides a comprehensive user study, design recommendations, and an automated evaluation benchmark for assessing LLM-based answer engines.

Findings

Frequent hallucinations in answer generation

Inaccurate source citations by answer engines

Variation in answer confidence levels across systems

Abstract

Large Language Model (LLM)-based applications are graduating from research prototypes to products serving millions of users, influencing how people write and consume information. A prominent example is the appearance of Answer Engines: LLM-based generative search engines supplanting traditional search engines. Answer engines not only retrieve relevant sources to a user query but synthesize answer summaries that cite the sources. To understand these systems' limitations, we first conducted a study with 21 participants, evaluating interactions with answer vs. traditional search engines and identifying 16 answer engine limitations. From these insights, we propose 16 answer engine design recommendations, linked to 8 metrics. An automated evaluation implementing our metrics on three popular engines (You.com, Perplexity.ai, BingChat) quantifies common limitations (e.g., frequent hallucination, inaccurate citation) and unique features (e.g., variation in answer confidence), with results mirroring user study insights. We release our Answer Engine Evaluation benchmark (AEE) to facilitate transparent evaluation of LLM-based applications.