Generative Retrieval Meets Multi-Graded Relevance

TL;DR

This paper introduces GR$^2$, a novel framework for generative retrieval that effectively handles multi-graded relevance by optimizing document identifiers and employing constrained contrastive training, improving retrieval accuracy.

Contribution

The paper extends generative retrieval to multi-graded relevance scenarios by developing a framework that ensures distinct, relevant identifiers and uses relevance-aware contrastive training.

Findings

GR$^2$ outperforms existing methods on multi-graded relevance datasets.

The approach effectively balances relevance and distinctness of document identifiers.

Experimental results show improved retrieval performance with multi-graded relevance data.

Abstract

Generative retrieval represents a novel approach to information retrieval. It uses an encoder-decoder architecture to directly produce relevant document identifiers (docids) for queries. While this method offers benefits, current approaches are limited to scenarios with binary relevance data, overlooking the potential for documents to have multi-graded relevance. Extending generative retrieval to accommodate multi-graded relevance poses challenges, including the need to reconcile likelihood probabilities for docid pairs and the possibility of multiple relevant documents sharing the same identifier. To address these challenges, we introduce a framework called GRaded Generative Retrieval (GR$^2$). GR$^2$ focuses on two key components: ensuring relevant and distinct identifiers, and implementing multi-graded constrained contrastive training. First, we create identifiers that are both semantically relevant and sufficiently distinct to represent individual documents effectively. This is achieved by jointly optimizing the relevance and distinctness of docids through a combination of docid generation and autoencoder models. Second, we incorporate information about the relationship between relevance grades to guide the training process. We use a constrained contrastive training strategy to bring the representations of queries and the identifiers of their relevant documents closer together, based on their respective relevance grades. Extensive experiments on datasets with both multi-graded and binary relevance demonstrate the effectiveness of GR$^2$.