A Combinatorial Identities Benchmark for Theorem Proving via Automated Theorem Generation

TL;DR

This paper introduces LeanComb, a new benchmark for formalizing combinatorial identities in Lean, and develops ATG4CI, an automated theorem generator that enhances theorem proving capabilities in combinatorics using LLMs and reinforcement learning.

Contribution

It presents the first formalized combinatorial identities benchmark in Lean and a novel automated theorem generator combining LLMs with reinforcement learning for tactic prediction.

Findings

Generated 260K theorems with formal proofs in Lean.

Models trained on this dataset improve tactic effectiveness.

Enhanced models achieve higher success rates in ATP for combinatorial identities.

Abstract

Large language models (LLMs) have significantly advanced formal theorem proving, yet the scarcity of high-quality training data constrains their capabilities in complex mathematical domains. Combinatorics, a cornerstone of mathematics, provides essential tools for analyzing discrete structures and solving optimization problems. However, its inherent complexity makes it particularly challenging for automated theorem proving (ATP) for combinatorial identities. To address this, we manually construct LeanComb, combinatorial identities benchmark in Lean, which is, to our knowledge, the first formalized theorem proving benchmark built for combinatorial identities. We develop an Automated Theorem Generator for Combinatorial Identities, ATG4CI, which combines candidate tactics suggested by a self-improving large language model with a Reinforcement Learning Tree Search approach for tactic prediction. By utilizing ATG4CI, we generate a LeanComb-Enhanced dataset comprising 260K combinatorial identities theorems, each with a complete formal proof in Lean, and experimental evaluations demonstrate that models trained on this dataset can generate more effective tactics, thereby improving success rates in automated theorem proving for combinatorial identities.