Empowering Multi-step Reasoning across Languages via Tree-of-Thoughts

TL;DR

This paper introduces Cross-lingual Tree-of-Thoughts (Cross-ToT), a novel prompting method that enhances multi-step reasoning across multiple languages in large language models, overcoming language barriers and improving performance.

Contribution

The paper presents Cross-ToT, a new cross-lingual reasoning approach inspired by Tree-of-Thoughts, enabling better multi-step reasoning in non-English languages.

Findings

Outperforms existing prompting methods in multi-language reasoning tasks.

Reduces the number of interactions needed for reasoning.

Achieves state-of-the-art performance across languages.

Abstract

Reasoning methods, best exemplified by the well-known Chain-of-Thought (CoT), empower the reasoning abilities of Large Language Models (LLMs) by eliciting them to solve complex tasks in a step-by-step manner. Although they are achieving significant success, the ability to deliver multi-step reasoning remains limited to English because of the imbalance in the distribution of pre-training data, which makes other languages a barrier. In this paper, we propose Cross-lingual Tree-of-Thoughts (Cross-ToT), a method for aligning Cross-lingual CoT reasoning across languages. The proposed method, through a self-consistent cross-lingual prompting mechanism inspired by the Tree-of-Thoughts approach, provides multi-step reasoning paths in different languages that, during the steps, lead to the final solution. Experimental evaluations show that our method significantly outperforms existing prompting methods by reducing the number of interactions and achieving state-of-the-art performance.