Annealed Co-Generation: Disentangling Variables via Progressive Pairwise Modeling

TL;DR

This paper introduces Annealed Co-Generation, a novel framework that uses pairwise diffusion models and an annealing process to efficiently generate multivariate scientific data while ensuring variable consistency.

Contribution

It proposes a low-dimensional pairwise diffusion modeling approach with an annealing process for multivariate generation, reducing computational complexity and improving coherence.

Findings

Effective on flow-field completion tasks

Successful antibody generation demonstration

Maintains high likelihood within pairwise distributions

Abstract

For multivariate co-generation in scientific applications, we advocate pairwise block rather than joint modeling of all variables. This design mitigates the computational burden and data imbalance. To this end, we propose an Annealed Co-Generation (ACG) framework that replaces high-dimensional diffusion modeling with a low-dimensional diffusion model, which enables multivariate co-generation by composing pairwise variable generations. We first train an unconditional diffusion model over causal variables that are disentangled into pairs. At inference time, we recover the joint distribution by coupling these pairwise models through shared common variables, enabling coherent multivariate generation without any additional training. By employing a three-stage annealing process-Consensus, Heating, and Cooling-our method enforces consistency across shared common variables and progressively constrains each pairwise data distribution to lie on a learnable manifold, while maintaining high likelihood within each pair. We demonstrate the framework's flexibility and efficacy on two distinct scientific tasks: flow-field completion and antibody generation. All datasets and code will be made publicly available upon publication.