STYLE: Improving Domain Transferability of Asking Clarification Questions in Large Language Model Powered Conversational Agents

TL;DR

This paper introduces Style, a novel method to improve the domain transferability of clarification questions in large language model-based conversational agents, significantly enhancing performance on unseen domains.

Contribution

The paper presents Style, a new approach that enhances domain transferability of clarification strategies in LLM-powered agents, addressing limitations of existing one-size-fits-all methods.

Findings

Style achieves ~10% improvement on four unseen domains.

It demonstrates strong domain transferability in experimental results.

The method outperforms existing strategies in diverse domain settings.

Abstract

Equipping a conversational search engine with strategies regarding when to ask clarification questions is becoming increasingly important across various domains. Attributing to the context understanding capability of LLMs and their access to domain-specific sources of knowledge, LLM-based clarification strategies feature rapid transfer to various domains in a post-hoc manner. However, they still struggle to deliver promising performance on unseen domains, struggling to achieve effective domain transferability. We take the first step to investigate this issue and existing methods tend to produce one-size-fits-all strategies across diverse domains, limiting their search effectiveness. In response, we introduce a novel method, called Style, to achieve effective domain transferability. Our experimental results indicate that Style bears strong domain transferability, resulting in an average search performance improvement of ~10% on four unseen domains.