Multilingual Datasets for Custom Input Extraction and Explanation Requests Parsing in Conversational XAI Systems

TL;DR

This paper introduces multilingual datasets and parsing strategies to improve custom input extraction and explanation requests understanding in conversational XAI systems, addressing data scarcity and multilingual challenges.

Contribution

It presents MultiCoXQL and Compass datasets for multilingual ConvXAI, along with a novel parsing approach and comprehensive evaluation of LLMs across multiple languages.

Findings

Multilingual datasets improve parsing performance in ConvXAI.

Cross-lingual transfer shows promising results for low-resource languages.

Evaluation reveals strengths and limitations of different LLMs in multilingual settings.

Abstract

Conversational explainable artificial intelligence (ConvXAI) systems based on large language models (LLMs) have garnered considerable attention for their ability to enhance user comprehension through dialogue-based explanations. Current ConvXAI systems often are based on intent recognition to accurately identify the user's desired intention and map it to an explainability method. While such methods offer great precision and reliability in discerning users' underlying intentions for English, a significant challenge in the scarcity of training data persists, which impedes multilingual generalization. Besides, the support for free-form custom inputs, which are user-defined data distinct from pre-configured dataset instances, remains largely limited. To bridge these gaps, we first introduce MultiCoXQL, a multilingual extension of the CoXQL dataset spanning five typologically diverse languages, including one low-resource language. Subsequently, we propose a new parsing approach aimed at enhancing multilingual parsing performance, and evaluate three LLMs on MultiCoXQL using various parsing strategies. Furthermore, we present Compass, a new multilingual dataset designed for custom input extraction in ConvXAI systems, encompassing 11 intents across the same five languages as MultiCoXQL. We conduct monolingual, cross-lingual, and multilingual evaluations on Compass, employing three LLMs of varying sizes alongside BERT-type models.