Rhetorical relations for information retrieval

TL;DR

This paper explores how rhetorical relations within texts, like contrast or cause, can improve information retrieval by modifying language models, leading to significant gains in retrieval effectiveness.

Contribution

It introduces a language model that incorporates rhetorical relations for relevance estimation, demonstrating improved IR performance over baseline models.

Findings

Certain rhetorical relations significantly enhance retrieval effectiveness.

The proposed model achieves over 10% improvement in mean average precision.

Empirical evaluation on TREC datasets validates the approach.

Abstract

Typically, every part in most coherent text has some plausible reason for its presence, some function that it performs to the overall semantics of the text. Rhetorical relations, e.g. contrast, cause, explanation, describe how the parts of a text are linked to each other. Knowledge about this socalled discourse structure has been applied successfully to several natural language processing tasks. This work studies the use of rhetorical relations for Information Retrieval (IR): Is there a correlation between certain rhetorical relations and retrieval performance? Can knowledge about a document's rhetorical relations be useful to IR? We present a language model modification that considers rhetorical relations when estimating the relevance of a document to a query. Empirical evaluation of different versions of our model on TREC settings shows that certain rhetorical relations can benefit retrieval effectiveness notably (> 10% in mean average precision over a state-of-the-art baseline).