Going beyond Compositions, DDPMs Can Produce Zero-Shot Interpolations

TL;DR

This paper demonstrates that DDPMs trained on separate data subsets can perform zero-shot interpolation, generating intermediate images without explicit training on those specific features.

Contribution

The study reveals that DDPMs can generate intermediate images in unexplored regions of the data distribution, beyond simple composition of learned factors.

Findings

DDPMs can interpolate between attributes like smiling and non-smiling faces.

Zero-shot interpolation is effective across different attributes and datasets.

Models trained on separate data subsets can generate images in untrained, intermediate regions.

Abstract

Denoising Diffusion Probabilistic Models (DDPMs) exhibit remarkable capabilities in image generation, with studies suggesting that they can generalize by composing latent factors learned from the training data. In this work, we go further and study DDPMs trained on strictly separate subsets of the data distribution with large gaps on the support of the latent factors. We show that such a model can effectively generate images in the unexplored, intermediate regions of the distribution. For instance, when trained on clearly smiling and non-smiling faces, we demonstrate a sampling procedure which can generate slightly smiling faces without reference images (zero-shot interpolation). We replicate these findings for other attributes as well as other datasets. Our code is available at https://github.com/jdeschena/ddpm-zero-shot-interpolation.