Towards Advanced Mathematical Reasoning for LLMs via First-Order Logic Theorem Proving

TL;DR

This paper introduces DREAM, a novel method to improve large language models' complex mathematical reasoning via first-order logic theorem proving, addressing their current limitations in multi-step deductions and proof strategies.

Contribution

DREAM enhances LLMs' mathematical reasoning with a new inference stage, diversification, and error feedback, leading to performance improvements and a curated theorem dataset.

Findings

Performance improved by 0.6% to 6.4% with DREAM

Curated dataset of 447 theorems in Lean 4 format

Addressed multi-step FOL reasoning challenges in LLMs

Abstract

Large language models (LLMs) have shown promising first-order logic (FOL) reasoning capabilities with applications in various areas. However, their effectiveness in complex mathematical reasoning involving multi-step FOL deductions is still under-researched. While LLMs perform competitively on established mathematical reasoning benchmarks, they struggle with multi-step FOL tasks, as demonstrated by Deepseek-Prover-V2-7B's low accuracy (4.2%) on our proposed theorem proving dataset. This issue arises from the limited exploration of diverse proof strategies and the potential for early reasoning mistakes to undermine entire proofs. To address these issues, we propose DREAM, a self-adaptive solution that enhances the Diversity and REAsonability of LLMs' generation strategies. DREAM incorporates an Axiom-Driven Strategy Diversification mechanism to promote varied strategic outcomes and a Sub-Proposition Error Feedback to help LLMs reflect on and correct their proofs. Our contributions include pioneering advancements in LLMs' mathematical reasoning through FOL theorem proving, introducing a novel inference stage solution that improves performance by 0.6% to 6.4%, and providing a curated dataset of 447 mathematical theorems in Lean 4 format for evaluation.