Scaling Neuro-symbolic Problem Solving: Solver-Free Learning of Constraints and Objectives

TL;DR

This paper presents a scalable neuro-symbolic architecture that learns to solve NP-hard reasoning problems directly from natural inputs, outperforming existing methods in training efficiency and solution quality.

Contribution

It introduces a differentiable probabilistic loss for learning constraints and objectives, enabling solver-free training and scalable inference for complex reasoning tasks.

Findings

Efficiently learns to solve NP-hard problems from natural inputs.

Requires less training time than other hybrid methods on Sudoku variants.

Optimizes regret better than Decision-Focused Learning on a visual Min-Cut/Max-Cut task.

Abstract

In the ongoing quest for hybridizing discrete reasoning with neural nets, there is an increasing interest in neural architectures that can learn how to solve discrete reasoning or optimization problems from natural inputs, a task that Large Language Models seem to struggle with. Objectives: We introduce a differentiable neuro-symbolic architecture and a loss function dedicated to learning how to solve NP-hard reasoning problems. Methods: Our new probabilistic loss allows for learning both the constraints and the objective, thus delivering a complete model that can be scrutinized and completed with side constraints. By pushing the combinatorial solver out of the training loop, our architecture also offers scalable training while exact inference gives access to maximum accuracy. Results: We empirically show that it can efficiently learn how to solve NP-hard reasoning problems from natural inputs. On three variants of the Sudoku benchmark -- symbolic, visual, and many-solution --, our approach requires a fraction of training time of other hybrid methods. On a visual Min-Cut/Max-cut task, it optimizes the regret better than a Decision-Focused-Learning regret-dedicated loss. Finally, it efficiently learns the energy optimization formulation of the large real-world problem of designing proteins.