Constrained Synthesis with Projected Diffusion Models

TL;DR

This paper presents a novel method to incorporate constraints into diffusion models, enabling them to generate data that complies with physical and other specified principles across diverse applications.

Contribution

It reformulates the diffusion sampling process as a constrained optimization problem, allowing for certified adherence to complex constraints during data synthesis.

Findings

Successfully applied to convex and non-convex constraints

Validated on material synthesis, physics-informed motion, and path planning

Achieved constraint satisfaction in diverse domains

Abstract

This paper introduces an approach to endow generative diffusion processes the ability to satisfy and certify compliance with constraints and physical principles. The proposed method recast the traditional sampling process of generative diffusion models as a constrained optimization problem, steering the generated data distribution to remain within a specified region to ensure adherence to the given constraints. These capabilities are validated on applications featuring both convex and challenging, non-convex, constraints as well as ordinary differential equations, in domains spanning from synthesizing new materials with precise morphometric properties, generating physics-informed motion, optimizing paths in planning scenarios, and human motion synthesis.