MultiplexNet: Towards Fully Satisfied Logical Constraints in Neural Networks

TL;DR

MultiplexNet introduces a method to incorporate expert domain knowledge as logical constraints into neural networks, ensuring 100% constraint satisfaction and often improving learning efficiency and solution quality.

Contribution

It presents a novel approach that encodes domain knowledge as logical formulas in DNF, enabling easy elicitation and integration into neural network training.

Findings

Achieves 100% constraint satisfaction in network outputs.

Learns to approximate distributions with fewer data samples.

Sometimes outperforms baseline methods in solution quality.

Abstract

We propose a novel way to incorporate expert knowledge into the training of deep neural networks. Many approaches encode domain constraints directly into the network architecture, requiring non-trivial or domain-specific engineering. In contrast, our approach, called MultiplexNet, represents domain knowledge as a logical formula in disjunctive normal form (DNF) which is easy to encode and to elicit from human experts. It introduces a Categorical latent variable that learns to choose which constraint term optimizes the error function of the network and it compiles the constraints directly into the output of existing learning algorithms. We demonstrate the efficacy of this approach empirically on several classical deep learning tasks, such as density estimation and classification in both supervised and unsupervised settings where prior knowledge about the domains was expressed as logical constraints. Our results show that the MultiplexNet approach learned to approximate unknown distributions well, often requiring fewer data samples than the alternative approaches. In some cases, MultiplexNet finds better solutions than the baselines; or solutions that could not be achieved with the alternative approaches. Our contribution is in encoding domain knowledge in a way that facilitates inference that is shown to be both efficient and general; and critically, our approach guarantees 100% constraint satisfaction in a network's output.