The Retrieval Effectiveness of Web Search Engines: Considering Results Descriptions

TL;DR

This study compares the retrieval effectiveness of five major web search engines, emphasizing the importance of result descriptions and revealing Google and Yahoo as top performers with Google providing more relevant descriptions.

Contribution

It introduces a systematic comparison of search engines considering both results and descriptions, proposing new retrieval measures and highlighting the significance of descriptions in user perception.

Findings

Google and Yahoo perform best overall.

Google provides significantly more relevant result descriptions.

Search engines should improve the relevance of descriptions.

Abstract

Purpose: To compare five major Web search engines (Google, Yahoo, MSN, Ask.com, and Seekport) for their retrieval effectiveness, taking into account not only the results but also the results descriptions. Design/Methodology/Approach: The study uses real-life queries. Results are made anonymous and are randomised. Results are judged by the persons posing the original queries. Findings: The two major search engines, Google and Yahoo, perform best, and there are no significant differences between them. Google delivers significantly more relevant result descriptions than any other search engine. This could be one reason for users perceiving this engine as superior. Research Limitations: The study is based on a user model where the user takes into account a certain amount of results rather systematically. This may not be the case in real life. Practical Implications: Implies that search engines should focus on relevant descriptions. Searchers are advised to use other search engines in addition to Google. Originality/Value: This is the first major study comparing results and descriptions systematically and proposes new retrieval measures to take into account results descriptions