Retrieval-Feedback-Driven Distillation and Preference Alignment for Efficient LLM-based Query Expansion

TL;DR

This paper introduces a retrieval-feedback-driven distillation framework that transfers retrieval-friendly query expansion behavior from large teacher models to smaller, more efficient student models, maintaining high retrieval performance while reducing inference costs.

Contribution

The paper presents a novel framework combining retrieval-feedback-driven distillation and preference alignment to create compact models with strong retrieval capabilities, applicable across multilingual settings.

Findings

The distilled Qwen3-4B model achieves about 97% of the teacher model's performance on DL19.

The approach effectively reduces inference costs while maintaining retrieval effectiveness.

Demonstrates strong practicality in both English and Chinese retrieval benchmarks.

Abstract

Large language models have recently enabled a generative paradigm for query expansion, but their high inference cost makes direct deployment difficult in practical retrieval systems. To address this issue, a retrieval-feedback-driven distillation and preference-alignment framework is proposed to transfer retrieval-friendly expansion behavior from a strong teacher model to a compact student model. Rather than relying on few-shot exemplars at inference time, the framework first leverages two complementary types of teacher-generated expansions, produced under zero-shot and few-shot prompting conditions, as supervision signals for distillation and as candidate pools for preference construction. A retrieval-metric-driven strategy is then introduced to automatically form chosen/rejected expansion pairs according to nDCG@10 differences, and Direct Preference Optimization is applied to explicitly align generation preferences with retrieval objectives. Experiments on TREC DL19/20/21 and MIRACL-zh show that the proposed approach preserves strong retrieval effectiveness while substantially reducing inference cost. In particular, the distilled Qwen3-4B model reaches about 97% of the teacher (DeepSeek-685B) model's nDCG@10 performance on DL19, and remains effective on the Chinese MIRACL-zh benchmark, demonstrating strong practicality across both English and Chinese retrieval settings.