On Multi-Step Theorem Prediction via Non-Parametric Structural Priors

TL;DR

This paper introduces a non-parametric, training-free approach for multi-step theorem prediction using structural priors, significantly improving reasoning accuracy and scalability in automated theorem proving.

Contribution

The authors propose Theorem Precedence Graphs and retrieval-augmented inference to address Structural Drift, enabling LLMs to perform structured reasoning without training.

Findings

Achieved 89.29% accuracy on FormalGeo7k benchmark.

Outperformed ICL baselines and matched supervised models.

Demonstrated effectiveness of explicit structural priors in reasoning.

Abstract

Multi-step theorem prediction is a central challenge in automated reasoning. Existing neural-symbolic approaches rely heavily on supervised parametric models, which exhibit limited generalization to evolving theorem libraries. In this work, we explore training-free theorem prediction through the lens of in-context learning (ICL). We identify a critical scalability bottleneck, termed Structural Drift: as reasoning depth increases, the performance of vanilla ICL degrades sharply, often collapsing to near zero. We attribute this failure to the LLM's inability to recover latent topological dependencies, leading to unstructured exploration. To address this issue, we propose Theorem Precedence Graphs, which encode temporal dependencies from historical solution traces as directed graphs, and impose explicit topological constraints that effectively prune the search space during inference. Coupled with retrieval-augmented graph construction and a stepwise symbolic executor, our approach enables LLMs to act as structured planners without any gradient-based optimization. Experiments on the FormalGeo7k benchmark show that our method achieves 89.29% accuracy, substantially outperforming ICL baselines and matching state-of-the-art supervised models. These results indicate that explicit structural priors offer a promising direction for scaling LLM-based symbolic reasoning.