GLEN: Generative Retrieval via Lexical Index Learning

TL;DR

GLEN introduces a novel generative retrieval method that learns lexical identifiers through a two-phase index learning strategy, achieving state-of-the-art performance on multiple benchmarks by directly generating document identifiers for queries.

Contribution

It proposes a new generative retrieval approach with a dynamic lexical index learning strategy and collision-free inference, addressing key challenges in existing methods.

Findings

Achieves state-of-the-art performance on NQ320k, MS MARCO, and BEIR datasets.

Effectively learns meaningful lexical identifiers and relevance signals.

Utilizes collision-free inference for efficient document ranking.

Abstract

Generative retrieval shed light on a new paradigm of document retrieval, aiming to directly generate the identifier of a relevant document for a query. While it takes advantage of bypassing the construction of auxiliary index structures, existing studies face two significant challenges: (i) the discrepancy between the knowledge of pre-trained language models and identifiers and (ii) the gap between training and inference that poses difficulty in learning to rank. To overcome these challenges, we propose a novel generative retrieval method, namely Generative retrieval via LExical iNdex learning (GLEN). For training, GLEN effectively exploits a dynamic lexical identifier using a two-phase index learning strategy, enabling it to learn meaningful lexical identifiers and relevance signals between queries and documents. For inference, GLEN utilizes collision-free inference, using identifier weights to rank documents without additional overhead. Experimental results prove that GLEN achieves state-of-the-art or competitive performance against existing generative retrieval methods on various benchmark datasets, e.g., NQ320k, MS MARCO, and BEIR. The code is available at https://github.com/skleee/GLEN.