Constraints-Guided Diffusion Reasoner for Neuro-Symbolic Learning

TL;DR

This paper introduces a diffusion-based neuro-symbolic reasoning framework that effectively learns logical constraints and improves symbolic reasoning in neural networks through a two-stage training process and policy optimization.

Contribution

It presents a novel diffusion reasoning approach with a two-stage training strategy and a Markov decision process formulation for enforcing logical constraints.

Findings

Achieves high accuracy on symbolic reasoning benchmarks.

Ensures logical consistency in neural network outputs.

Outperforms existing methods in Sudoku and maze tasks.

Abstract

Enabling neural networks to learn complex logical constraints and fulfill symbolic reasoning is a critical challenge. Bridging this gap often requires guiding the neural network's output distribution to move closer to the symbolic constraints. While diffusion models have shown remarkable generative capability across various domains, we employ the powerful architecture to perform neuro-symbolic learning and solve logical puzzles. Our diffusion-based pipeline adopts a two-stage training strategy: the first stage focuses on cultivating basic reasoning abilities, while the second emphasizes systematic learning of logical constraints. To impose hard constraints on neural outputs in the second stage, we formulate the diffusion reasoner as a Markov decision process and innovatively fine-tune it with an improved proximal policy optimization algorithm. We utilize a rule-based reward signal derived from the logical consistency of neural outputs and adopt a flexible strategy to optimize the diffusion reasoner's policy. We evaluate our methodology on some classical symbolic reasoning benchmarks, including Sudoku, Maze, pathfinding and preference learning. Experimental results demonstrate that our approach achieves outstanding accuracy and logical consistency among neural networks.