A Semantic Loss Function for Deep Learning with Symbolic Knowledge

TL;DR

This paper introduces a semantic loss function that integrates symbolic knowledge into deep learning, improving semi-supervised classification and structured output prediction.

Contribution

It presents a novel semantic loss function derived from logical constraints, enhancing deep learning models with symbolic reasoning capabilities.

Findings

Achieves near-state-of-the-art results on semi-supervised multi-class classification.

Significantly improves prediction of structured objects like rankings and paths.

Demonstrates effective integration of symbolic reasoning with deep learning.

Abstract

This paper develops a novel methodology for using symbolic knowledge in deep learning. From first principles, we derive a semantic loss function that bridges between neural output vectors and logical constraints. This loss function captures how close the neural network is to satisfying the constraints on its output. An experimental evaluation shows that it effectively guides the learner to achieve (near-)state-of-the-art results on semi-supervised multi-class classification. Moreover, it significantly increases the ability of the neural network to predict structured objects, such as rankings and paths. These discrete concepts are tremendously difficult to learn, and benefit from a tight integration of deep learning and symbolic reasoning methods.