The retrieval effectiveness of search engines on navigational queries

TL;DR

This study evaluates the effectiveness of major search engines in handling navigational queries, revealing that Google, Yahoo, and MSN perform best with around 90% success in retrieving the correct homepage.

Contribution

It provides a systematic comparison of major search engines' performance on navigational queries, highlighting differences and emphasizing the importance of this aspect in search engine design.

Findings

Google, Yahoo, and MSN answer about 90% of navigational queries correctly.

Ask and Exalead perform worse but still achieve good scores.

Results are specific to German-language queries and interfaces.

Abstract

Purpose - To test major Web search engines on their performance on navigational queries, i.e. searches for homepages. Design/methodology/approach - 100 real user queries are posed to six search engines (Google, Yahoo, MSN, Ask, Seekport, and Exalead). Users described the desired pages, and the results position of these is recorded. Measured success N and mean reciprocal rank are calculated. Findings - Performance of the major search engines Google, Yahoo, and MSN is best, with around 90 percent of queries answered correctly. Ask and Exalead perform worse but receive good scores as well. Research limitations/implications - All queries were in German, and the German-language interfaces of the search engines were used. Therefore, the results are only valid for German queries. Practical implications - When designing a search engine to compete with the major search engines, care should be taken on the performance on navigational queries. Users can be influenced easily in their quality ratings of search engines based on this performance. Originality/value - This study systematically compares the major search engines on navigational queries and compares the findings with studies on the retrieval effectiveness of the engines on informational queries. Paper type - research paper