TensorLog: A Differentiable Deductive Database

TL;DR

TensorLog introduces a differentiable probabilistic deductive database that integrates logical reasoning with gradient-based learning by converting logical clauses into factor graphs and unrolling belief propagation into differentiable functions.

Contribution

The paper presents TensorLog, a novel framework that enables differentiable reasoning in logical theories, allowing integration with deep learning systems.

Findings

Efficient linear-time compilation and inference methods.

Ability to perform inference in complex logical theories.

Demonstrated compatibility with gradient-based learning systems.

Abstract

Large knowledge bases (KBs) are useful in many tasks, but it is unclear how to integrate this sort of knowledge into "deep" gradient-based learning systems. To address this problem, we describe a probabilistic deductive database, called TensorLog, in which reasoning uses a differentiable process. In TensorLog, each clause in a logical theory is first converted into certain type of factor graph. Then, for each type of query to the factor graph, the message-passing steps required to perform belief propagation (BP) are "unrolled" into a function, which is differentiable. We show that these functions can be composed recursively to perform inference in non-trivial logical theories containing multiple interrelated clauses and predicates. Both compilation and inference in TensorLog are efficient: compilation is linear in theory size and proof depth, and inference is linear in database size and the number of message-passing steps used in BP. We also present experimental results with TensorLog and discuss its relationship to other first-order probabilistic logics.