Unsupervised Decomposition and Recombination with Discriminator-Driven Diffusion Models

TL;DR

This paper introduces a discriminator-driven diffusion model that learns factorized latent representations without supervision, enabling improved compositional generation and diverse robotic video synthesis.

Contribution

It proposes an adversarial training approach to enhance latent factor discovery and recombination quality in diffusion models without requiring factor-level labels.

Findings

Outperforms prior methods on CelebA-HQ, Virtual KITTI, CLEVR, and Falcor3D

Achieves lower FID scores and better disentanglement metrics (MIG, MCC)

Generates diverse robotic trajectories increasing exploration coverage

Abstract

Decomposing complex data into factorized representations can reveal reusable components and enable synthesizing new samples via component recombination. We investigate this in the context of diffusion-based models that learn factorized latent spaces without factor-level supervision. In images, factors can capture background, illumination, and object attributes; in robotic videos, they can capture reusable motion components. To improve both latent factor discovery and quality of compositional generation, we introduce an adversarial training signal via a discriminator trained to distinguish between single-source samples and those generated by recombining factors across sources. By optimizing the generator to fool this discriminator, we encourage physical and semantic consistency in the resulting recombinations. Our method outperforms implementations of prior baselines on CelebA-HQ, Virtual KITTI, CLEVR, and Falcor3D, achieving lower FID scores and better disentanglement as measured by MIG and MCC. Furthermore, we demonstrate a novel application to robotic video trajectories: by recombining learned action components, we generate diverse sequences that significantly increase state-space coverage for exploration on the LIBERO benchmark.