ConQX: Semantic Expansion of Spoken Queries for Intent Detection based on Conditioned Text Generation

TL;DR

This paper introduces ConQX, a semantic expansion method for spoken queries using GPT-2, which improves intent detection by enriching noisy, short queries through conditioned text generation and fine-tuning of BERT and RoBERTa models.

Contribution

The paper presents a novel approach combining conditioned text generation with prompt mining to enhance intent detection in spoken queries.

Findings

Semantic expansion improves intent detection accuracy.

GPT-2-based generation effectively enriches noisy queries.

Fine-tuning BERT and RoBERTa on expanded queries yields better results.

Abstract

Intent detection of spoken queries is a challenging task due to their noisy structure and short length. To provide additional information regarding the query and enhance the performance of intent detection, we propose a method for semantic expansion of spoken queries, called ConQX, which utilizes the text generation ability of an auto-regressive language model, GPT-2. To avoid off-topic text generation, we condition the input query to a structured context with prompt mining. We then apply zero-shot, one-shot, and few-shot learning. We lastly use the expanded queries to fine-tune BERT and RoBERTa for intent detection. The experimental results show that the performance of intent detection can be improved by our semantic expansion method.