RL-based Query Rewriting with Distilled LLM for online E-Commerce Systems
Duy A. Nguyen, Rishi Kesav Mohan, Van Yang, Pritom Saha Akash, Kevin Chen-Chuan Chang

TL;DR
This paper introduces a hybrid RL-based query rewriting method using distilled LLMs for online e-commerce search, balancing efficiency and quality to improve relevance and adaptability.
Contribution
It presents a novel hybrid pipeline combining offline knowledge distillation and online reinforcement learning with LLM-simulated feedback for improved query rewriting.
Findings
Significant improvement in query relevance and diversity.
Enhanced adaptability in dynamic e-commerce environments.
Efficient online inference with reduced latency and cost.
Abstract
Query rewriting (QR) is a critical technique in e-commerce search, addressing the lexical gap between user queries and product descriptions to enhance search performance. Existing QR approaches typically fall into two categories: discriminative models and generative methods leveraging large language models (LLMs). Discriminative models often struggle with natural language understanding and offer limited flexibility in rewriting, while generative LLMs, despite producing high-quality rewrites, face high inference latency and cost in online settings. These limitations force offline deployment, making them vulnerable to issues like information staleness and semantic drift. To overcome these challenges, we propose a novel hybrid pipeline for QR that balances efficiency and effectiveness. Our approach combines offline knowledge distillation to create a lightweight but efficient student model…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSemantic Web and Ontologies · Advanced Database Systems and Queries · Data Mining Algorithms and Applications
MethodsKnowledge Distillation
