Intent Detection for code-mix utterances in task oriented dialogue systems

TL;DR

This paper evaluates various models and vector representations for intent detection in code-mixed and multilingual dialogue systems, aiming to identify the most effective combination for real-world multi-language applications.

Contribution

It systematically compares models and vectorization techniques for intent detection in code-mixed and multilingual utterances, filling a gap in existing single-language focused research.

Findings

Certain model-vector combinations outperform others in code-mixed contexts.

Performance varies significantly across datasets with different language compositions.

The study provides insights into scalable intent detection for multilingual dialogue systems.

Abstract

Intent detection is an essential component of task oriented dialogue systems. Over the years, extensive research has been conducted resulting in many state of the art models directed towards resolving user's intents in dialogue. A variety of vector representations foruser utterances have been explored for the same. However, these models and vectorization approaches have more so been evaluated in a single language environment. Dialogude systems generally have to deal with queries in different languages. We thus conduct experiments across combinations of models and various vectors representations for Code Mix as well as multi language utterances and evaluate how these models scale to a multi language environment. Our aim is to find the best suitable combination of vector representation and models for the process of intent detection for Code Mix utterances. we have evaluated the experiments on two different datasets consisting of only Code Mix utterances and the other dataset consisting of English, Hindi and Code Mix English Hindi utterances.