Semantic-Enhanced Differentiable Search Index Inspired by Learning Strategies

TL;DR

This paper introduces a Semantic-Enhanced Differentiable Search Index (SE-DSI) that incorporates human learning-inspired strategies to improve document retrieval by using meaningful descriptions and semantic features for better memorization.

Contribution

The paper proposes a novel SE-DSI model that enhances traditional DSI with elaborative descriptions and semantic features inspired by cognitive psychology learning strategies.

Findings

Improved retrieval performance over baseline models

Effective use of semantic descriptions for document identifiers

Enhanced memorization of document associations

Abstract

Recently, a new paradigm called Differentiable Search Index (DSI) has been proposed for document retrieval, wherein a sequence-to-sequence model is learned to directly map queries to relevant document identifiers. The key idea behind DSI is to fully parameterize traditional ``index-retrieve'' pipelines within a single neural model, by encoding all documents in the corpus into the model parameters. In essence, DSI needs to resolve two major questions: (1) how to assign an identifier to each document, and (2) how to learn the associations between a document and its identifier. In this work, we propose a Semantic-Enhanced DSI model (SE-DSI) motivated by Learning Strategies in the area of Cognitive Psychology. Our approach advances original DSI in two ways: (1) For the document identifier, we take inspiration from Elaboration Strategies in human learning. Specifically, we assign each document an Elaborative Description based on the query generation technique, which is more meaningful than a string of integers in the original DSI; and (2) For the associations between a document and its identifier, we take inspiration from Rehearsal Strategies in human learning. Specifically, we select fine-grained semantic features from a document as Rehearsal Contents to improve document memorization. Both the offline and online experiments show improved retrieval performance over prevailing baselines.