Konkani LLM: Multi-Script Instruction Tuning and Evaluation for a Low-Resource Indian Language

TL;DR

This paper introduces Konkani LLM, a series of fine-tuned models for the low-resource Konkani language, utilizing synthetic instruction data and multi-script evaluation to improve performance across diverse orthographies.

Contribution

The paper presents the creation of Konkani-Instruct-100k dataset, development of Konkani LLM models, and a Multi-Script Konkani Benchmark for cross-script evaluation, addressing low-resource challenges.

Findings

Konkani LLM outperforms base models in translation tasks.

Models achieve competitive results against proprietary baselines.

Synthetic instruction tuning improves low-resource language performance.

Abstract

Large Language Models (LLMs) consistently under perform in low-resource linguistic contexts such as Konkani. This performance deficit stems from acute training data scarcity compounded by high script diversity across Devanagari, Romi and Kannada orthographies. To address this gap, we introduce Konkani-Instruct-100k, a comprehensive synthetic instruction-tuning dataset generated through Gemini 3. We establish rigorous baseline benchmarks by evaluating leading open-weights architectures including Llama 3.1, Qwen2.5 and Gemma 3 alongside proprietary closed-source models. Our primary contribution involves the development of Konkani LLM, a series of fine-tuned models optimized for regional nuances. Furthermore, we are developing the Multi-Script Konkani Benchmark to facilitate cross-script linguistic evaluation. In machine translation, Konkani LLM delivers consistent gains over the corresponding base models and is competitive with and in several settings surpasses proprietary baselines