Intent Recognition and Out-of-Scope Detection using LLMs in Multi-party Conversations

TL;DR

This paper presents a hybrid method combining BERT and LLMs for intent recognition and out-of-scope detection in multi-party conversations, effective in zero and few-shot settings, reducing data annotation needs.

Contribution

It introduces a novel hybrid approach leveraging BERT and LLMs for improved intent recognition and OOS detection in multi-party dialogues with minimal data.

Findings

Sharing BERT outputs with LLMs improves performance.

Effective in zero and few-shot learning scenarios.

Reduces reliance on large annotated datasets.

Abstract

Intent recognition is a fundamental component in task-oriented dialogue systems (TODS). Determining user intents and detecting whether an intent is Out-of-Scope (OOS) is crucial for TODS to provide reliable responses. However, traditional TODS require large amount of annotated data. In this work we propose a hybrid approach to combine BERT and LLMs in zero and few-shot settings to recognize intents and detect OOS utterances. Our approach leverages LLMs generalization power and BERT's computational efficiency in such scenarios. We evaluate our method on multi-party conversation corpora and observe that sharing information from BERT outputs to LLMs leads to system performance improvement.