ProductAgent: Benchmarking Conversational Product Search Agent with Asking Clarification Questions

TL;DR

This paper presents ProductAgent, a conversational agent designed for product search that asks clarification questions to improve accuracy, supported by a new benchmark PROCLARE and experiments with a user simulator.

Contribution

Introduction of ProductAgent, a novel conversational product search agent with strategic clarification capabilities, and the PROCLARE benchmark for evaluation.

Findings

ProductAgent improves retrieval accuracy with more dialogue turns.

The agent effectively asks clarification questions to refine user demands.

Experiments demonstrate positive user interaction and performance gains.

Abstract

This paper introduces the task of product demand clarification within an e-commercial scenario, where the user commences the conversation with ambiguous queries and the task-oriented agent is designed to achieve more accurate and tailored product searching by asking clarification questions. To address this task, we propose ProductAgent, a conversational information seeking agent equipped with abilities of strategic clarification question generation and dynamic product retrieval. Specifically, we develop the agent with strategies for product feature summarization, query generation, and product retrieval. Furthermore, we propose the benchmark called PROCLARE to evaluate the agent's performance both automatically and qualitatively with the aid of a LLM-driven user simulator. Experiments show that ProductAgent interacts positively with the user and enhances retrieval performance with increasing dialogue turns, where user demands become gradually more explicit and detailed. All the source codes will be released after the review anonymity period.