Methods for comparing rankings of search engine results

TL;DR

This paper introduces various measures for comparing search engine result rankings, applying them to daily data over two periods for multiple engines and queries to analyze consistency and changes.

Contribution

It presents new measures for ranking comparison and applies them to real search engine data over time, offering insights into ranking stability and differences.

Findings

Search engine rankings vary daily and over time.

Different search engines produce significantly different rankings.

Rankings show some stability but also notable fluctuations.

Abstract

In this paper we present a number of measures that compare rankings of search engine results. We apply these measures to five queries that were monitored daily for two periods of about 21 days each. Rankings of the different search engines (Google, Yahoo and Teoma for text searches and Google, Yahoo and Picsearch for image searches) are compared on a daily basis, in addition to longitudinal comparisons of the same engine for the same query over time. The results and rankings of the two periods are compared as well.