AutoConv: Automatically Generating Information-seeking Conversations with Large Language Models

TL;DR

AutoConv leverages large language models to generate high-quality synthetic information-seeking conversations, reducing reliance on scarce human-annotated data and improving performance on key datasets.

Contribution

AutoConv introduces a novel method for synthetic conversation generation using LLMs, addressing data scarcity in information-seeking dialogue research.

Findings

AutoConv outperforms strong baselines on two datasets.

It significantly reduces dependence on human annotation.

Experimental results demonstrate substantial quality improvements.

Abstract

Information-seeking conversation, which aims to help users gather information through conversation, has achieved great progress in recent years. However, the research is still stymied by the scarcity of training data. To alleviate this problem, we propose AutoConv for synthetic conversation generation, which takes advantage of the few-shot learning ability and generation capacity of large language models (LLM). Specifically, we formulate the conversation generation problem as a language modeling task, then finetune an LLM with a few human conversations to capture the characteristics of the information-seeking process and use it for generating synthetic conversations with high quality. Experimental results on two frequently-used datasets verify that AutoConv has substantial improvements over strong baselines and alleviates the dependence on human annotation. In addition, we also provide several analysis studies to promote future research.