Structured Reasoning with Tree-of-Thoughts for Bengali Math Word Problems

TL;DR

This paper explores Tree-of-Thought reasoning for Bengali math word problems, demonstrating that it improves accuracy over Chain-of-Thought, especially in larger language models, and offers a promising approach for low-resource languages.

Contribution

It systematically evaluates Tree-of-Thought reasoning for Bengali MWPs, showing its effectiveness over CoT in improving accuracy with large language models.

Findings

ToT improves accuracy by up to 5 percentage points over CoT.

Larger models benefit more from ToT than smaller ones.

ToT provides more reliable reasoning in low-resource language NLP.

Abstract

Mathematical Word Problems (MWPs) are among the most challenging tasks in natural language processing because they require both linguistic understanding and multi-step numerical reasoning. While Chain-of-Thought (CoT) prompting has shown promise, its linear structure often propagates errors, limiting overall effectiveness. To address this limitation, we present the a systematic study of Tree-of-Thought (ToT) reasoning for Bengali MWPs using the SOMADHAN dataset. Owing to computational and token-cost constraints, we evaluate a curated set of 100 representative problems across multiple large language models (LLMs), including GPT-OSS and LLaMA variants, under standard prompting, CoT, and ToT strategies. Our results show that CoT improves baseline accuracy from 78% (standard prompting) to 83% on average, while ToT further increases performance by up to 5 percentage points, achieving 88% accuracy with GPT-OSS-120B. These improvements highlight that ToT is particularly effective in medium-to-large-scale models but may offer less advantage for smaller ones. Overall, our findings establish ToT as a robust framework for solving mathematical problems in low-resource languages such as Bengali. More broadly, this study shows that structured reasoning methods like ToT can provide more reliable and globally consistent outcomes than CoT, paving the way for better reasoning strategies in multilingual NLP.