Fixed versus Dynamic Co-Occurrence Windows in TextRank Term Weights for Information Retrieval

TL;DR

This paper investigates the impact of using fixed versus dynamic co-occurrence windows in TextRank for IR, demonstrating that dynamic windows aligned with document structure improve early search precision.

Contribution

It introduces a method for dynamically adjusting co-occurrence windows in TextRank based on sentence and paragraph boundaries, enhancing IR performance.

Findings

Dynamic co-occurrence windows improve early precision in IR

Adjusting co-occurrence context benefits term weighting

TextRank with dynamic windows outperforms fixed window approaches

Abstract

TextRank is a variant of PageRank typically used in graphs that represent documents, and where vertices denote terms and edges denote relations between terms. Quite often the relation between terms is simple term co-occurrence within a fixed window of k terms. The output of TextRank when applied iteratively is a score for each vertex, i.e. a term weight, that can be used for information retrieval (IR) just like conventional term frequency based term weights. So far, when computing TextRank term weights over co- occurrence graphs, the window of term co-occurrence is al- ways ?xed. This work departs from this, and considers dy- namically adjusted windows of term co-occurrence that fol- low the document structure on a sentence- and paragraph- level. The resulting TextRank term weights are used in a ranking function that re-ranks 1000 initially returned search results in order to improve the precision of the ranking. Ex- periments with two IR collections show that adjusting the vicinity of term co-occurrence when computing TextRank term weights can lead to gains in early precision.