A Survey of Multilingual Reasoning in Language Models

TL;DR

This survey reviews the current state of multilingual reasoning in language models, discussing methods, challenges, benchmarks, and future directions for improving reasoning across diverse languages.

Contribution

It provides the first comprehensive overview of multilingual reasoning in language models, including methods, datasets, evaluation benchmarks, and analysis of performance.

Findings

Multilingual reasoning in LMs is still in early development.

Existing benchmarks reveal significant performance gaps across languages.

Future research should focus on handling low-resource languages and complex reasoning tasks.

Abstract

While reasoning and multilingual capabilities in language models (LMs) have achieved remarkable progress in recent years, their integration into a unified paradigm - multilingual reasoning - is at a nascent stage. Multilingual reasoning requires language models to handle logical reasoning across languages while addressing misalignment, biases, and challenges in low-resource settings. This survey provides the first in-depth review of multilingual reasoning in LMs. In this survey, we provide a systematic overview of existing methods that leverage LMs for multilingual reasoning, specifically outlining the challenges, motivations, and foundational aspects of applying language models to reason across diverse languages. We provide an overview of the standard data resources used for training multilingual reasoning in LMs and the evaluation benchmarks employed to assess their multilingual capabilities. Next, we analyze various state-of-the-art methods and their performance on these benchmarks. Finally, we explore future research opportunities to improve multilingual reasoning in LMs, focusing on enhancing their ability to handle diverse languages and complex reasoning tasks. Rapid growth of evolving developments in this field can be actively tracked on our project page: [https://github.com/AkashGhosh/Survey-of-Multilingual-Reasoning-in-Language-Models](https://github.com/AkashGhosh/Survey-of-Multilingual-Reasoning-in-Language-Models)