Deep Generative Models with Hard Linear Equality Constraints

TL;DR

This paper introduces a probabilistically sound method for integrating hard linear equality constraints into deep generative models, ensuring constraint compliance while maintaining high data quality across diverse datasets.

Contribution

The authors develop gradient estimators enabling differentiable learning of constrained distributions in DGMs, outperforming existing heuristic approaches in constraint satisfaction and generative performance.

Findings

Our method guarantees constraint satisfaction in generated data.

It achieves superior generative quality compared to existing strategies.

The approach is validated across multiple datasets and scientific applications.

Abstract

While deep generative models~(DGMs) have demonstrated remarkable success in capturing complex data distributions, they consistently fail to learn constraints that encode domain knowledge and thus require constraint integration. Existing solutions to this challenge have primarily relied on heuristic methods and often ignore the underlying data distribution, harming the generative performance. In this work, we propose a probabilistically sound approach for enforcing the hard constraints into DGMs to generate constraint-compliant and realistic data. This is achieved by our proposed gradient estimators that allow the constrained distribution, the data distribution conditioned on constraints, to be differentiably learned. We carry out extensive experiments with various DGM model architectures over five image datasets and three scientific applications in which domain knowledge is governed by linear equality constraints. We validate that the standard DGMs almost surely generate data violating the constraints. Among all the constraint integration strategies, ours not only guarantees the satisfaction of constraints in generation but also archives superior generative performance than the other methods across every benchmark.