GraphMind: Theorem Selection and Conclusion Generation Framework with Dynamic GNN for LLM Reasoning

TL;DR

GraphMind introduces a dynamic graph-based framework combining GNNs and LLMs to enhance multi-step reasoning through explicit theorem selection and conclusion generation.

Contribution

It presents a novel heterogeneous evolving graph model that improves context-aware reasoning and interpretability in LLM-based multi-step reasoning tasks.

Findings

Achieves significant performance improvements on QA datasets.

Outperforms existing baselines in multi-step reasoning.

Demonstrates the effectiveness of dynamic graph modeling for reasoning.

Abstract

Large language models (LLMs) have demonstrated impressive capabilities in natural language understanding and generation, including multi-step reasoning such as mathematical proving. However, existing approaches often lack an explicit and dynamic mechanism to structurally represent and evolve intermediate reasoning states, which limits their ability to perform context-aware theorem selection and iterative conclusion generation. To address these challenges, we propose GraphMind, a novel dynamic graph-based framework that integrates the graph neural network (GNN) with LLMs to iteratively select theorems and generate intermediate conclusions for multi-step reasoning. Our method models the reasoning process as a heterogeneous evolving graph, where nodes represent conditions, theorems, and conclusions, while edges capture logical dependencies between nodes. By encoding the current reasoning state with GNN and leveraging semantic matching for theorem selection, our framework enables context-aware, interpretable, and structured reasoning in a closed-loop manner. Experiments on various question-answering (QA) datasets demonstrate that our proposed GraphMind method achieves consistent performance improvements and significantly outperforms existing baselines in multi-step reasoning, validating the effectiveness and generalizability of our approach.