Harnessing Deep Neural Networks with Logic Rules

TL;DR

This paper presents a framework that integrates first-order logic rules into deep neural networks, improving their interpretability and performance across tasks like sentiment analysis and named entity recognition.

Contribution

It introduces an iterative distillation method to embed logic rules into neural network weights, enhancing flexibility and accuracy.

Findings

Achieved state-of-the-art results on sentiment analysis.

Improved named entity recognition performance.

Demonstrated effectiveness with only a few simple rules.

Abstract

Combining deep neural networks with structured logic rules is desirable to harness flexibility and reduce uninterpretability of the neural models. We propose a general framework capable of enhancing various types of neural networks (e.g., CNNs and RNNs) with declarative first-order logic rules. Specifically, we develop an iterative distillation method that transfers the structured information of logic rules into the weights of neural networks. We deploy the framework on a CNN for sentiment analysis, and an RNN for named entity recognition. With a few highly intuitive rules, we obtain substantial improvements and achieve state-of-the-art or comparable results to previous best-performing systems.