Scaling the Vocabulary of Non-autoregressive Models for Efficient Generative Retrieval

TL;DR

This paper introduces PIXAR, a non-autoregressive model with an expanded vocabulary for efficient generative retrieval, significantly improving retrieval performance while maintaining low latency and cost.

Contribution

PIXAR expands NAR model vocabularies to include multi-word entities, reducing token dependencies and enhancing retrieval accuracy without increasing inference latency.

Findings

31.0% improvement in MRR@10 on MS MARCO

23.2% increase in Hits@5 on Natural Questions

5.08% increase in ad clicks in online experiments

Abstract

Generative Retrieval introduces a new approach to Information Retrieval by reframing it as a constrained generation task, leveraging recent advancements in Autoregressive (AR) language models. However, AR-based Generative Retrieval methods suffer from high inference latency and cost compared to traditional dense retrieval techniques, limiting their practical applicability. This paper investigates fully Non-autoregressive (NAR) language models as a more efficient alternative for generative retrieval. While standard NAR models alleviate latency and cost concerns, they exhibit a significant drop in retrieval performance (compared to AR models) due to their inability to capture dependencies between target tokens. To address this, we question the conventional choice of limiting the target token space to solely words or sub-words. We propose PIXAR, a novel approach that expands the target vocabulary of NAR models to include multi-word entities and common phrases (up to 5 million tokens), thereby reducing token dependencies. PIXAR employs inference optimization strategies to maintain low inference latency despite the significantly larger vocabulary. Our results demonstrate that PIXAR achieves a relative improvement of 31.0% in MRR@10 on MS MARCO and 23.2% in Hits@5 on Natural Questions compared to standard NAR models with similar latency and cost. Furthermore, online A/B experiments on a large commercial search engine show that PIXAR increases ad clicks by 5.08% and revenue by 4.02%.