Large Language Models for Math Education in Low-Resource Languages: A Study in Sinhala and Tamil

TL;DR

This study evaluates the mathematical reasoning capabilities of large language models in Sinhala and Tamil, revealing language-dependent performance gaps and emphasizing the importance of language-specific assessments for educational AI deployment.

Contribution

It introduces a bilingual dataset authored by native speakers and systematically assesses LLMs' math reasoning across languages, highlighting the variability in model performance.

Findings

Basic arithmetic reasoning transfers well across Sinhala, Tamil, and English.

Complex reasoning tasks show significant performance degradation in Sinhala and Tamil.

Performance varies by model and problem type, indicating language-specific challenges.

Abstract

Large language models (LLMs) have achieved strong results in mathematical reasoning, and are increasingly deployed as tutoring and learning support tools in educational settings. However, their reliability for students working in non-English languages, especially low-resource languages, remains poorly understood. We examine this gap by evaluating mathematical reasoning in Sinhala and Tamil -- two languages widely used in South Asian schools but underrepresented in artificial intelligence (AI) research. Using a taxonomy of six math problem types, from basic arithmetic to complex unit conflict and optimization problems, we evaluate four prominent large language models. To avoid translation artifacts that confound language ability with translation quality, we construct a parallel dataset in which each problem is independently authored in Sinhala and Tamil by native speakers, and in English by fluent speakers, all with strong mathematical backgrounds. Our analysis demonstrates that while basic arithmetic reasoning transfers robustly across languages, complex reasoning tasks show significant degradation in Tamil and Sinhala. The pattern of failures varies by model and problem type, suggesting that strong performance in English does not guarantee reliable performance across languages. These findings have direct implications for the deployment of AI tools in multilingual classrooms, and highlight the need for language-specific evaluation before adopting large language models as math tutoring aids in non-English educational contexts.