NeuralFastLAS: Fast Logic-Based Learning from Raw Data

TL;DR

NeuralFastLAS is a scalable, end-to-end neuro-symbolic learning method that efficiently trains neural networks with symbolic rule learning, achieving state-of-the-art accuracy and significantly faster training times.

Contribution

It introduces a novel approach that jointly trains neural networks and symbolic learners, with a rule posterior distribution to enhance training stability.

Findings

Achieves state-of-the-art accuracy on arithmetic and logical tasks.

Training time is up to 100 times faster than existing methods.

Provides theoretical guarantees for correctness of the symbolic solutions.

Abstract

Symbolic rule learners generate interpretable solutions, however they require the input to be encoded symbolically. Neuro-symbolic approaches overcome this issue by mapping raw data to latent symbolic concepts using a neural network. Training the neural and symbolic components jointly is difficult, due to slow and unstable learning, hence many existing systems rely on hand-engineered rules to train the network. We introduce NeuralFastLAS, a scalable and fast end-to-end approach that trains a neural network jointly with a symbolic learner. For a given task, NeuralFastLAS computes a relevant set of rules, proved to contain an optimal symbolic solution, trains a neural network using these rules, and finally finds an optimal symbolic solution to the task while taking network predictions into account. A key novelty of our approach is learning a posterior distribution on rules while training the neural network to improve stability during training. We provide theoretical results for a sufficient condition on network training to guarantee correctness of the final solution. Experimental results demonstrate that NeuralFastLAS is able to achieve state-of-the-art accuracy in arithmetic and logical tasks, with a training time that is up to two orders of magnitude faster than other jointly trained neuro-symbolic methods.