Cross-lingual Prompting: Improving Zero-shot Chain-of-Thought Reasoning across Languages

TL;DR

This paper introduces cross-lingual prompting techniques to enhance zero-shot chain-of-thought reasoning across multiple languages, achieving state-of-the-art results and addressing language generalization limitations.

Contribution

It proposes cross-lingual alignment and task-specific solver prompting, along with self-consistent prompting, to improve reasoning in multilingual contexts.

Findings

CLP and CLSP outperform existing prompting methods.

Achieve state-of-the-art performance on several benchmarks.

Enhance reasoning accuracy across multiple languages.

Abstract

Chain-of-thought (CoT) is capable of eliciting models to explicitly generate reasoning paths, thus promoting reasoning accuracy and attracting increasing attention. Specifically, zero-shot CoT achieves remarkable improvements in a wide range of reasoning tasks by simply instructing the LLM with the prompt "Let's think step by step!". Despite the success of zero-shot CoT, the existing zero-shot prompting techniques remain limited to a single language, making it challenging to generalize to other languages and hindering global development. In this work, we introduce cross-lingual prompting (CLP), aiming to improve zero-shot CoT reasoning across languages. Specifically, CLP consists of two main components: (1) cross-lingual alignment prompting and (2) task-specific solver prompting. The cross-lingual alignment prompting is responsible for aligning representations across different languages, whereas the task-specific solver prompting is used to generate the final chain of thoughts and results for the reasoning task. In addition, we further introduce cross-lingual self-consistent prompting (CLSP) to ensemble different reasoning paths across languages. Our experimental evaluations on several benchmarks demonstrate that CLP and CLSP significantly outperform the existing prompting methods and achieve state-of-the-art performance. We hope this work will inspire further breakthroughs in cross-lingual CoT.