RL-based Query Rewriting with Distilled LLM for online E-Commerce Systems

TL;DR

This paper introduces a hybrid RL-based query rewriting method using distilled LLMs for online e-commerce search, balancing efficiency and quality to improve relevance and adaptability.

Contribution

It presents a novel hybrid pipeline combining offline knowledge distillation and online reinforcement learning with LLM-simulated feedback for improved query rewriting.

Findings

Significant improvement in query relevance and diversity.

Enhanced adaptability in dynamic e-commerce environments.

Efficient online inference with reduced latency and cost.

Abstract

Query rewriting (QR) is a critical technique in e-commerce search, addressing the lexical gap between user queries and product descriptions to enhance search performance. Existing QR approaches typically fall into two categories: discriminative models and generative methods leveraging large language models (LLMs). Discriminative models often struggle with natural language understanding and offer limited flexibility in rewriting, while generative LLMs, despite producing high-quality rewrites, face high inference latency and cost in online settings. These limitations force offline deployment, making them vulnerable to issues like information staleness and semantic drift. To overcome these challenges, we propose a novel hybrid pipeline for QR that balances efficiency and effectiveness. Our approach combines offline knowledge distillation to create a lightweight but efficient student model with online reinforcement learning (RL) to refine query rewriting dynamically using real-time feedback. A key innovation is the use of LLMs as simulated human feedback, enabling scalable reward signals and cost-effective evaluation without manual annotations. Experimental results on Amazon ESCI dataset demonstrate significant improvements in query relevance, diversity, and adaptability, as well as positive feedback from the LLM simulation. This work contributes to advancing LLM capabilities for domain-specific applications, offering a robust solution for dynamic and complex e-commerce search environments.