Modeling Perceived Relevance for Tail Queries without Click-Through Data

TL;DR

This paper proposes a method to model perceived relevance for tail queries in web search without click data by designing snippet features, improving relevance prediction and search accuracy for less popular queries.

Contribution

It introduces a novel set of snippet features to estimate perceived relevance without relying on click data, benefiting tail query search performance.

Findings

Effective in predicting perceived relevance of search results.

Improves search accuracy for tail queries without click data.

Snippet features enhance ranking quality for unpopular queries.

Abstract

Click-through data has been used in various ways in Web search such as estimating relevance between documents and queries. Since only search snippets are perceived by users before issuing any clicks, the relevance induced by clicks are usually called \emph{perceived relevance} which has proven to be quite useful for Web search. While there is plenty of click data for popular queries, very little information is available for unpopular tail ones. These tail queries take a large portion of the search volume but search accuracy for these queries is usually unsatisfactory due to data sparseness such as limited click information. In this paper, we study the problem of modeling perceived relevance for queries without click-through data. Instead of relying on users' click data, we carefully design a set of snippet features and use them to approximately capture the perceived relevance. We study the effectiveness of this set of snippet features in two settings: (1) predicting perceived relevance and (2) enhancing search engine ranking. Experimental results show that our proposed model is effective to predict the relative perceived relevance of Web search results. Furthermore, our proposed snippet features are effective to improve search accuracy for longer tail queries without click-through data.