Multilingual Mathematical Reasoning: Advancing Open-Source LLMs in Hindi and English

TL;DR

This paper enhances the mathematical reasoning abilities of open-source multilingual LLMs in Hindi and English through curriculum learning, decomposition strategies, and structured solutions, achieving significant performance improvements.

Contribution

Introduces a novel Decomposition Strategy and Structured Solution Design for training multilingual LLMs in mathematical reasoning, with empirical validation on multiple models and datasets.

Findings

WizardMath 7B outperforms Gemini on English datasets by +6% accuracy.

Bilingual training achieves results comparable to single-language models.

Model improvements demonstrate effective reasoning in Hindi and English.

Abstract

Large Language Models (LLMs) excel in linguistic tasks but struggle with mathematical reasoning, particularly in non English languages like Hindi. This research aims to enhance the mathematical reasoning skills of smaller, resource efficient open-source LLMs in both Hindi and English. We evaluate models like OpenHathi 7B, LLaMA-2 7B, WizardMath 7B, Mistral 7B, LLeMMa 7B, MAmmoTH 7B, Gemini Pro, and GPT-4 using zero-shot, few-shot chain-of-thought (CoT) methods, and supervised fine-tuning. Our approach incorporates curriculum learning, progressively training models on increasingly difficult problems, a novel Decomposition Strategy to simplify complex arithmetic operations, and a Structured Solution Design that divides solutions into phases. Our experiments result in notable performance enhancements. WizardMath 7B exceeds Gemini's accuracy on English datasets by +6% and matches Gemini's performance on Hindi datasets. Adopting a bilingual approach that combines English and Hindi samples achieves results comparable to individual language models, demonstrating the capability to learn mathematical reasoning in both languages. This research highlights the potential for improving mathematical reasoning in open-source LLMs.