On Synthetic Data Strategies for Domain-Specific Generative Retrieval

TL;DR

This paper explores synthetic data generation techniques for training domain-specific generative retrieval models, focusing on query creation and hard negative mining to improve scalability and relevance detection.

Contribution

It introduces a two-stage training framework utilizing LLM-generated queries and hard negative mining, advancing domain-specific generative retrieval methods.

Findings

Synthetic data strategies improve retrieval performance.

LLM-generated queries capture nuanced relevancy signals.

Hard negative mining enhances document ranking accuracy.

Abstract

This paper investigates synthetic data generation strategies in developing generative retrieval models for domain-specific corpora, thereby addressing the scalability challenges inherent in manually annotating in-domain queries. We study the data strategies for a two-stage training framework: in the first stage, which focuses on learning to decode document identifiers from queries, we investigate LLM-generated queries across multiple granularity (e.g. chunks, sentences) and domain-relevant search constraints that can better capture nuanced relevancy signals. In the second stage, which aims to refine document ranking through preference learning, we explore the strategies for mining hard negatives based on the initial model's predictions. Experiments on public datasets over diverse domains demonstrate the effectiveness of our synthetic data generation and hard negative sampling approach.