Generate then Refine: Data Augmentation for Zero-shot Intent Detection

TL;DR

This paper introduces a two-stage data augmentation approach using a large language model and a smaller refiner to improve zero-shot intent detection in new domains, outperforming existing methods.

Contribution

It presents a novel two-step method combining zero-shot LLM generation and a refiner model to enhance intent detection data quality in unseen domains.

Findings

Refiner improves data diversity and utility over zero-shot baseline.

The method achieves better intent classification accuracy on real data.

Two-step approach outperforms common baseline data augmentation methods.

Abstract

In this short paper we propose a data augmentation method for intent detection in zero-resource domains. Existing data augmentation methods rely on few labelled examples for each intent category, which can be expensive in settings with many possible intents. We use a two-stage approach: First, we generate utterances for intent labels using an open-source large language model in a zero-shot setting. Second, we develop a smaller sequence-to-sequence model (the Refiner), to improve the generated utterances. The Refiner is fine-tuned on seen domains and then applied to unseen domains. We evaluate our method by training an intent classifier on the generated data, and evaluating it on real (human) data. We find that the Refiner significantly improves the data utility and diversity over the zero-shot LLM baseline for unseen domains and over common baseline approaches. Our results indicate that a two-step approach of a generative LLM in zero-shot setting and a smaller sequence-to-sequence model can provide high-quality data for intent detection.