Learning Term Weights for Ad-hoc Retrieval

TL;DR

This paper introduces a learning-based approach to compute term weights for ad-hoc retrieval, moving beyond traditional heuristics and probabilistic models by leveraging learning-to-rank techniques.

Contribution

It proposes a novel method to learn term weights directly from data, improving relevance scoring in information retrieval systems.

Findings

Demonstrates improved retrieval performance over traditional models

Introduces a data-driven approach to term weighting

Validates effectiveness on benchmark datasets

Abstract

Most Information Retrieval models compute the relevance score of a document for a given query by summing term weights specific to a document or a query. Heuristic approaches, like TF-IDF, or probabilistic models, like BM25, are used to specify how a term weight is computed. In this paper, we propose to leverage learning-to-rank principles to learn how to compute a term weight for a given document based on the term occurrence pattern.