Manifold-Aware Perturbations for Constrained Generative Modeling

TL;DR

This paper introduces a novel, efficient method for modifying distributions in constrained generative models, ensuring support matches ambient space and respects manifold geometry, improving data recovery and sampling stability.

Contribution

It presents a new perturbation technique for equality-constrained generative models that is computationally efficient, mathematically justified, and adaptable to various models.

Findings

Improves data distribution recovery in constrained models

Enables stable sampling with diffusion models and normalizing flows

Supports manifold-aware distribution modifications

Abstract

Generative models have enjoyed widespread success in a variety of applications. However, they encounter inherent mathematical limitations in modeling distributions where samples are constrained by equalities, as is frequently the setting in scientific domains. In this work, we develop a computationally cheap, mathematically justified, and highly flexible distributional modification for combating known pitfalls in equality-constrained generative models. We propose perturbing the data distribution in a constraint-aware way such that the new distribution has support matching the ambient space dimension while still implicitly incorporating underlying manifold geometry. Through theoretical analyses and empirical evidence on several representative tasks, we illustrate that our approach consistently enables data distribution recovery and stable sampling with both diffusion models and normalizing flows.