EXPLAIN, AGREE, LEARN: Scaling Learning for Neural Probabilistic Logic

TL;DR

This paper introduces EXAL, a scalable neuro-symbolic learning method that uses sampling-based objectives with theoretical error bounds, outperforming previous methods on complex tasks like MNIST addition and Warcraft pathfinding.

Contribution

The paper proposes a novel sampling-based learning objective with proven error bounds and introduces the EXAL method, enabling scalable neuro-symbolic learning with theoretical guarantees.

Findings

EXAL outperforms recent NeSy methods on large-scale problems.

The sampling objective's error diminishes with increased samples and sample diversity.

Theoretical analysis confirms the method's scalability and accuracy.

Abstract

Neural probabilistic logic systems follow the neuro-symbolic (NeSy) paradigm by combining the perceptive and learning capabilities of neural networks with the robustness of probabilistic logic. Learning corresponds to likelihood optimization of the neural networks. However, to obtain the likelihood exactly, expensive probabilistic logic inference is required. To scale learning to more complex systems, we therefore propose to instead optimize a sampling based objective. We prove that the objective has a bounded error with respect to the likelihood, which vanishes when increasing the sample count. Furthermore, the error vanishes faster by exploiting a new concept of sample diversity. We then develop the EXPLAIN, AGREE, LEARN (EXAL) method that uses this objective. EXPLAIN samples explanations for the data. AGREE reweighs each explanation in concordance with the neural component. LEARN uses the reweighed explanations as a signal for learning. In contrast to previous NeSy methods, EXAL can scale to larger problem sizes while retaining theoretical guarantees on the error. Experimentally, our theoretical claims are verified and EXAL outperforms recent NeSy methods when scaling up the MNIST addition and Warcraft pathfinding problems.