Quality Over Clicks: Intrinsic Quality-Driven Iterative Reinforcement Learning for Cold-Start E-Commerce Query Suggestion

TL;DR

This paper introduces Cold-EQS, an iterative reinforcement learning framework designed for cold-start e-commerce query suggestion, leveraging quality-based rewards to improve suggestions without relying on click data.

Contribution

The paper presents Cold-EQS, a novel reinforcement learning approach that optimizes query suggestions using quality metrics, addressing cold-start challenges in e-commerce systems.

Findings

Achieves +6.81% improvement in online chatUV.

Provides an extensive EQS-Benchmark with 16,949 queries.

Demonstrates strong correlation between offline and online effectiveness.

Abstract

Existing dialogue systems rely on Query Suggestion (QS) to enhance user engagement. Recent efforts typically employ large language models with Click-Through Rate (CTR) model, yet fail in cold-start scenarios due to their heavy reliance on abundant online click data for effective CTR model training. To bridge this gap, we propose Cold-EQS, an iterative reinforcement learning framework for Cold-Start E-commerce Query Suggestion (EQS). Specifically, we leverage answerability, factuality, and information gain as reward to continuously optimize the quality of suggested queries. To continuously optimize our QS model, we estimate uncertainty for grouped candidate suggested queries to select hard and ambiguous samples from online user queries lacking click signals. In addition, we provide an EQS-Benchmark comprising 16,949 online user queries for offline training and evaluation. Extensive offline and online experiments consistently demonstrate a strong positive correlation between online and offline effectiveness. Both offline and online experimental results demonstrate the superiority of our Cold-EQS, achieving a significant +6.81% improvement in online chatUV.