Efficient Multilingual Dialogue Processing via Translation Pipelines and Distilled Language Models

TL;DR

This paper introduces a multilingual dialogue system utilizing translation pipelines and a distilled language model, achieving high performance across nine languages in summarization and question answering tasks without task-specific fine-tuning.

Contribution

It presents a novel three-stage translation and distillation approach that enables effective multilingual dialogue processing with compact models.

Findings

Achieved high win rates in NLPAI4Health 2025 tasks

Demonstrated strong performance on Marathi, Tamil, and Hindi

Showed effectiveness of translation-based methods for low-resource languages

Abstract

This paper presents team Kl33n3x's multilingual dialogue summarization and question answering system developed for the NLPAI4Health 2025 shared task. The approach employs a three-stage pipeline: forward translation from Indic languages to English, multitask text generation using a 2.55B parameter distilled language model, and reverse translation back to source languages. By leveraging knowledge distillation techniques, this work demonstrates that compact models can achieve highly competitive performance across nine languages. The system achieved strong win rates across the competition's tasks, with particularly robust performance on Marathi (86.7% QnA), Tamil (86.7% QnA), and Hindi (80.0% QnA), demonstrating the effectiveness of translation-based approaches for low-resource language processing without task-specific fine-tuning.