Ranking of Wikipedia articles in search engines revisited: Fair ranking for reasonable quality?
Dirk Lewandowski, Ulrike Spree

TL;DR
This study investigates whether Wikipedia article rankings in search engines reflect their actual quality, proposing a heuristic evaluation method and analyzing its correlation with relevance judgments and user trust.
Contribution
The paper introduces a heuristic evaluation method for Wikipedia quality and compares it with search engine relevance judgments, highlighting trust and systemic shortcomings.
Findings
Wikipedia results are judged better than other sources in search rankings.
Relevance judgments often align with heuristic evaluations.
High trust in Wikipedia explains discrepancies between relevance and quality scores.
Abstract
This paper aims to review the fiercely discussed question of whether the ranking of Wikipedia articles in search engines is justified by the quality of the articles. After an overview of current research on information quality in Wikipedia, a summary of the extended discussion on the quality of encyclopedic entries in general is given. On this basis, a heuristic method for evaluating Wikipedia entries is developed and applied to Wikipedia articles that scored highly in a search engine retrieval effectiveness test and compared with the relevance judgment of jurors. In all search engines tested, Wikipedia results are unanimously judged better by the jurors than other results on the corresponding results position. Relevance judgments often roughly correspond with the results from the heuristic evaluation. Cases in which high relevance judgments are not in accordance with the comparatively…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
