Distribution Matching via Generalized Consistency Models

TL;DR

This paper introduces a new distribution matching method inspired by consistency models, combining the advantages of CNF and GANs, with theoretical validation and experimental results on synthetic and real datasets.

Contribution

The paper proposes a novel distribution matching approach that leverages consistency models, offering a simpler objective and greater flexibility compared to traditional GANs.

Findings

The proposed model effectively matches distributions in synthetic datasets.

The method demonstrates competitive performance on real-world data.

Theoretical analysis confirms the validity of the objective function.

Abstract

Recent advancement in generative models have demonstrated remarkable performance across various data modalities. Beyond their typical use in data synthesis, these models play a crucial role in distribution matching tasks such as latent variable modeling, domain translation, and domain adaptation. Generative Adversarial Networks (GANs) have emerged as the preferred method of distribution matching due to their efficacy in handling high-dimensional data and their flexibility in accommodating various constraints. However, GANs often encounter challenge in training due to their bi-level min-max optimization objective and susceptibility to mode collapse. In this work, we propose a novel approach for distribution matching inspired by the consistency models employed in Continuous Normalizing Flow (CNF). Our model inherits the advantages of CNF models, such as having a straight forward norm minimization objective, while remaining adaptable to different constraints similar to GANs. We provide theoretical validation of our proposed objective and demonstrate its performance through experiments on synthetic and real-world datasets.