DeepLogic: Towards End-to-End Differentiable Logical Reasoning

TL;DR

This paper investigates how neural networks can learn to perform logical reasoning over rule-based knowledge by representing symbolic logic in high-dimensional spaces, using a new dataset and analyzing learned representations.

Contribution

It introduces a novel approach to end-to-end differentiable logical reasoning with neural networks and provides insights into how reasoning algorithms are internally represented.

Findings

Neural networks can learn to infer logical entailment from rule-based programs.

Representations of atoms, literals, and rules emerge during training.

Performance varies with complexity and length of logical expressions.

Abstract

Combining machine learning with logic-based expert systems in order to get the best of both worlds are becoming increasingly popular. However, to what extent machine learning can already learn to reason over rule-based knowledge is still an open problem. In this paper, we explore how symbolic logic, defined as logic programs at a character level, is learned to be represented in a high-dimensional vector space using RNN-based iterative neural networks to perform reasoning. We create a new dataset that defines 12 classes of logic programs exemplifying increased level of complexity of logical reasoning and train the networks in an end-to-end fashion to learn whether a logic program entails a given query. We analyse how learning the inference algorithm gives rise to representations of atoms, literals and rules within logic programs and evaluate against increasing lengths of predicate and constant symbols as well as increasing steps of multi-hop reasoning.