pix2rule: End-to-end Neuro-symbolic Rule Learning

TL;DR

This paper introduces pix2rule, an end-to-end neuro-symbolic system that processes images to learn objects, relations, and rules, outperforming existing models in symbolic reasoning and relational learning tasks.

Contribution

It presents a differentiable layer enabling symbolic rule extraction within a deep learning framework, advancing neuro-symbolic integration for visual reasoning.

Findings

Outperforms state-of-the-art symbolic learners

Scales beyond existing deep relational neural networks

Successfully learns rules from visual data

Abstract

Humans have the ability to seamlessly combine low-level visual input with high-level symbolic reasoning often in the form of recognising objects, learning relations between them and applying rules. Neuro-symbolic systems aim to bring a unifying approach to connectionist and logic-based principles for visual processing and abstract reasoning respectively. This paper presents a complete neuro-symbolic method for processing images into objects, learning relations and logical rules in an end-to-end fashion. The main contribution is a differentiable layer in a deep learning architecture from which symbolic relations and rules can be extracted by pruning and thresholding. We evaluate our model using two datasets: subgraph isomorphism task for symbolic rule learning and an image classification domain with compound relations for learning objects, relations and rules. We demonstrate that our model scales beyond state-of-the-art symbolic learners and outperforms deep relational neural network architectures.