LogicMP: A Neuro-symbolic Approach for Encoding First-order Logic Constraints

TL;DR

LogicMP introduces a neural layer that efficiently encodes first-order logic constraints into neural networks, enabling better performance across diverse tasks by leveraging mean-field inference over Markov Logic Networks.

Contribution

The paper presents a novel neural layer, LogicMP, that performs mean-field variational inference over MLNs, integrating FOL constraints into neural networks with improved efficiency and modularity.

Findings

Outperforms competitors in graph, image, and text tasks.

Reduces inference complexity from sequential to parallel tensor operations.

Demonstrates theoretical mitigation of MLN inference difficulty.

Abstract

Integrating first-order logic constraints (FOLCs) with neural networks is a crucial but challenging problem since it involves modeling intricate correlations to satisfy the constraints. This paper proposes a novel neural layer, LogicMP, whose layers perform mean-field variational inference over an MLN. It can be plugged into any off-the-shelf neural network to encode FOLCs while retaining modularity and efficiency. By exploiting the structure and symmetries in MLNs, we theoretically demonstrate that our well-designed, efficient mean-field iterations effectively mitigate the difficulty of MLN inference, reducing the inference from sequential calculation to a series of parallel tensor operations. Empirical results in three kinds of tasks over graphs, images, and text show that LogicMP outperforms advanced competitors in both performance and efficiency.