Unexpected Knowledge: Auditing Wikipedia and Grokipedia Search Recommendations

TL;DR

This study compares search recommendation behaviors on Wikipedia and Grokipedia, revealing that both platforms often produce unexpected and weakly related results, with notable differences in content distribution and exploration trajectories.

Contribution

First comparative analysis of search engine recommendations on Wikipedia and Grokipedia, highlighting their similarities and differences in content surfacing and user exploration.

Findings

Both platforms generate weakly related and unexpected results.

Significant differences in topical distribution and content categories.

Search results evolve systematically over multiple exploration stages.

Abstract

Encyclopedic knowledge platforms are key gateways through which users explore information online. The recent release of Grokipedia, a fully AI-generated encyclopedia, introduces a new alternative to traditional, well-established platforms like Wikipedia. In this context, search engine mechanisms play an important role in guiding users exploratory paths, yet their behavior across different encyclopedic systems remains underexplored. In this work, we address this gap by providing the first comparative analysis of search engine in Wikipedia and Grokipedia. Using nearly 10,000 neutral English words and their substrings as queries, we collect over 70,000 search engine results and examine their semantic alignment, overlap, and topical structure. We find that both platforms frequently generate results that are weakly related to the original query and, in many cases, surface unexpected content starting from innocuous queries. Despite these shared properties, the two systems often produce substantially different recommendation sets for the same query. Through topical annotation and trajectory analysis, we further identify systematic differences in how content categories are surfaced and how search engine results evolve over multiple stages of exploration. Overall, our findings show that unexpected search engine outcomes are a common feature of both the platforms, even though they exhibit discrepancies in terms of topical distribution and query suggestions.