ILLUMINER: Instruction-tuned Large Language Models as Few-shot Intent Classifier and Slot Filler

TL;DR

ILLUMINER leverages instruction-tuned large language models to improve intent classification and slot filling with fewer examples, outperforming existing methods and reducing training data needs.

Contribution

The paper introduces ILLUMINER, a novel approach framing IC and SF as language generation tasks using instruction-tuned models, with a more efficient prompting method and effective fine-tuning.

Findings

Outperforms state-of-the-art joint IC+SF methods.

Achieves 11.1--32.2 percentage points improvement in slot filling.

Requires less than 6% of training data for comparable performance.

Abstract

State-of-the-art intent classification (IC) and slot filling (SF) methods often rely on data-intensive deep learning models, limiting their practicality for industry applications. Large language models on the other hand, particularly instruction-tuned models (Instruct-LLMs), exhibit remarkable zero-shot performance across various natural language tasks. This study evaluates Instruct-LLMs on popular benchmark datasets for IC and SF, emphasizing their capacity to learn from fewer examples. We introduce ILLUMINER, an approach framing IC and SF as language generation tasks for Instruct-LLMs, with a more efficient SF-prompting method compared to prior work. A comprehensive comparison with multiple baselines shows that our approach, using the FLAN-T5 11B model, outperforms the state-of-the-art joint IC+SF method and in-context learning with GPT3.5 (175B), particularly in slot filling by 11.1--32.2 percentage points. Additionally, our in-depth ablation study demonstrates that parameter-efficient fine-tuning requires less than 6% of training data to yield comparable performance with traditional full-weight fine-tuning.