Improved and Efficient Conversational Slot Labeling through Question Answering

TL;DR

This paper introduces a novel QA-based approach to slot labeling in dialog NLU, leveraging pretrained language models and lightweight adapters to improve performance, especially in low-data scenarios, and demonstrates state-of-the-art results.

Contribution

The paper presents a new QA reformulation of slot labeling, utilizing PLMs with adapters and larger synthetic datasets to enhance efficiency and accuracy in dialog NLU tasks.

Findings

Achieved state-of-the-art slot labeling performance with QA-tuned PLMs.

Significant improvements in low-data regimes.

QA-based models reach a performance ceiling in high-data settings.

Abstract

Transformer-based pretrained language models (PLMs) offer unmatched performance across the majority of natural language understanding (NLU) tasks, including a body of question answering (QA) tasks. We hypothesize that improvements in QA methodology can also be directly exploited in dialog NLU; however, dialog tasks must be \textit{reformatted} into QA tasks. In particular, we focus on modeling and studying \textit{slot labeling} (SL), a crucial component of NLU for dialog, through the QA optics, aiming to improve both its performance and efficiency, and make it more effective and resilient to working with limited task data. To this end, we make a series of contributions: 1) We demonstrate how QA-tuned PLMs can be applied to the SL task, reaching new state-of-the-art performance, with large gains especially pronounced in such low-data regimes. 2) We propose to leverage contextual information, required to tackle ambiguous values, simply through natural language. 3) Efficiency and compactness of QA-oriented fine-tuning are boosted through the use of lightweight yet effective adapter modules. 4) Trading-off some of the quality of QA datasets for their size, we experiment with larger automatically generated QA datasets for QA-tuning, arriving at even higher performance. Finally, our analysis suggests that our novel QA-based slot labeling models, supported by the PLMs, reach a performance ceiling in high-data regimes, calling for more challenging and more nuanced benchmarks in future work.