Projected Coupled Diffusion for Test-Time Constrained Joint Generation

TL;DR

This paper introduces Projected Coupled Diffusion (PCD), a test-time method for jointly generating correlated samples from multiple diffusion models while enforcing constraints efficiently.

Contribution

The paper proposes a novel framework that couples guidance and projection steps to enable constrained joint generation without retraining models.

Findings

Effective in image-pair generation, object manipulation, and robot motion planning.

Improves coupling effects and guarantees constraint satisfaction.

Operates with low additional computational cost.

Abstract

Modifications to test-time sampling have emerged as an important extension to diffusion algorithms, with the goal of biasing the generative process to achieve a given objective without having to retrain the entire diffusion model. However, generating jointly correlated samples from multiple pre-trained diffusion models while simultaneously enforcing task-specific constraints without costly retraining has remained challenging. To this end, we propose Projected Coupled Diffusion (PCD), a novel test-time framework for constrained joint generation. PCD introduces a coupled guidance term into the generative dynamics to encourage coordination between diffusion models and incorporates a projection step at each diffusion step to enforce hard constraints. Empirically, we demonstrate the effectiveness of PCD in application scenarios of image-pair generation, object manipulation, and multi-robot motion planning. Our results show improved coupling effects and guaranteed constraint satisfaction without incurring excessive computational costs.