Chain-of-Reasoning: Towards Unified Mathematical Reasoning in Large Language Models via a Multi-Paradigm Perspective

TL;DR

This paper introduces a unified framework called Chain-of-Reasoning (CoR) that combines multiple reasoning paradigms in large language models to improve mathematical problem-solving and generalization.

Contribution

The paper presents CoR, a multi-paradigm reasoning framework with a novel training strategy, significantly enhancing LLMs' mathematical reasoning capabilities across diverse tasks.

Findings

CoR-Math-7B outperforms SOTA models in theorem proving.

Achieves 41.0% improvement over GPT-4o on the MATH benchmark.

Demonstrates strong zero-shot generalization in mathematical reasoning.

Abstract

Large Language Models (LLMs) have made notable progress in mathematical reasoning, yet often rely on single-paradigm reasoning, limiting their effectiveness across diverse tasks. We introduce Chain-of-Reasoning (CoR), a novel unified framework integrating multiple reasoning paradigms--Natural Language Reasoning (NLR), Algorithmic Reasoning (AR), and Symbolic Reasoning (SR)--to enable synergistic collaboration. CoR generates multiple potential answers via different reasoning paradigms and synthesizes them into a coherent final solution. We propose a Progressive Paradigm Training (PPT) strategy for models to progressively master these paradigms, leading to CoR-Math-7B. Experimental results demonstrate that CoR-Math-7B significantly outperforms current SOTA models, achieving up to a 41.0% absolute improvement over GPT-4o in theorem proving and a 15.0% improvement over RL-based methods on the MATH benchmark in arithmetic tasks. These results show the enhanced mathematical comprehension ability of our model, enabling zero-shot generalization across tasks.