Semantic Loss Functions for Neuro-Symbolic Structured Prediction

TL;DR

This paper introduces semantic loss functions that incorporate structured symbolic knowledge into neural network training, improving structured prediction by enforcing output dependencies and enabling complex domain synthesis.

Contribution

The paper proposes a novel semantic loss approach that injects symbolic structure into neural training, enhances it with entropy minimization, and integrates with generative models for complex object synthesis.

Findings

Semantic loss improves structured prediction accuracy.

Entropy minimization enhances structure adherence.

Integration with GANs enables synthesis of structured objects.

Abstract

Structured output prediction problems are ubiquitous in machine learning. The prominent approach leverages neural networks as powerful feature extractors, otherwise assuming the independence of the outputs. These outputs, however, jointly encode an object, e.g. a path in a graph, and are therefore related through the structure underlying the output space. We discuss the semantic loss, which injects knowledge about such structure, defined symbolically, into training by minimizing the network's violation of such dependencies, steering the network towards predicting distributions satisfying the underlying structure. At the same time, it is agnostic to the arrangement of the symbols, and depends only on the semantics expressed thereby, while also enabling efficient end-to-end training and inference. We also discuss key improvements and applications of the semantic loss. One limitations of the semantic loss is that it does not exploit the association of every data point with certain features certifying its membership in a target class. We should therefore prefer minimum-entropy distributions over valid structures, which we obtain by additionally minimizing the neuro-symbolic entropy. We empirically demonstrate the benefits of this more refined formulation. Moreover, the semantic loss is designed to be modular and can be combined with both discriminative and generative neural models. This is illustrated by integrating it into generative adversarial networks, yielding constrained adversarial networks, a novel class of deep generative models able to efficiently synthesize complex objects obeying the structure of the underlying domain.