Rejection Sampling IMLE: Designing Priors for Better Few-Shot Image Synthesis

TL;DR

This paper introduces RS-IMLE, a novel method that modifies the prior distribution in IMLE to improve few-shot image synthesis, achieving higher quality results than existing models.

Contribution

It proposes a new prior distribution for IMLE that addresses latent code correspondence issues, enhancing few-shot image generation performance.

Findings

RS-IMLE outperforms existing methods on nine datasets.

Significant quality improvements in generated images.

Addresses latent code mismatch problem in IMLE.

Abstract

An emerging area of research aims to learn deep generative models with limited training data. Prior generative models like GANs and diffusion models require a lot of data to perform well, and their performance degrades when they are trained on only a small amount of data. A recent technique called Implicit Maximum Likelihood Estimation (IMLE) has been adapted to the few-shot setting, achieving state-of-the-art performance. However, current IMLE-based approaches encounter challenges due to inadequate correspondence between the latent codes selected for training and those drawn during inference. This results in suboptimal test-time performance. We theoretically show a way to address this issue and propose RS-IMLE, a novel approach that changes the prior distribution used for training. This leads to substantially higher quality image generation compared to existing GAN and IMLE-based methods, as validated by comprehensive experiments conducted on nine few-shot image datasets.