Semi-Supervised Generative Learning via Latent Space Distribution Matching

TL;DR

LSDM is a new semi-supervised generative framework that learns a low-dimensional latent space and matches joint distributions using Wasserstein distance, improving generation quality with limited paired data.

Contribution

The paper introduces LSDM, a two-stage semi-supervised generative model that leverages unpaired data and provides theoretical bounds and insights connecting to latent diffusion models.

Findings

LSDM outperforms existing methods in class-conditional generation.

Unpaired data improves geometric fidelity of generated images.

LSDM offers a unified statistical perspective on latent-space generative models.

Abstract

We introduce Latent Space Distribution Matching (LSDM), a novel framework for semi-supervised generative modeling of conditional distributions. LSDM operates in two stages: (i) learning a low-dimensional latent space from both paired and unpaired data, and (ii) performing joint distribution matching in this space via the 1-Wasserstein distance, using only paired data. This two-step approach minimizes an upper bound on the 1-Wasserstein distance between joint distributions, reducing reliance on scarce paired samples while enabling fast one-step generation. Theoretically, we establish non-asymptotic error bounds and demonstrate a key benefit of unpaired data: enhanced geometric fidelity in generated outputs. Furthermore, by extending the scope of its two core steps, LSDM provides a coherent statistical perspective that connects to a broad class of latent-space approaches. Notably, Latent Diffusion Models (LDMs) can be viewed as a variant of LSDM, in which joint distribution matching is achieved indirectly via score matching. Consequently, our results also provide theoretical insights into the consistency of LDMs. Empirical evaluations on real-world image tasks, including class-conditional generation and image super-resolution, demonstrate the effectiveness of LSDM in leveraging unpaired data to enhance generation quality.