Few-Shot Image Generation by Conditional Relaxing Diffusion Inversion

TL;DR

This paper introduces CRDI, a training-free diffusion-based method for few-shot image generation that enhances diversity and reduces overfitting without fine-tuning, outperforming GAN-based methods and matching state-of-the-art results.

Contribution

CRDI is a novel training-free approach that reconstructs target images and expands diversity via a guidance embedding and perturbation scheduler, avoiding fine-tuning.

Findings

CRDI surpasses GAN-based reconstruction techniques.

CRDI achieves state-of-the-art performance in FSIG.

CRDI mitigates overfitting and catastrophic forgetting.

Abstract

In the field of Few-Shot Image Generation (FSIG) using Deep Generative Models (DGMs), accurately estimating the distribution of target domain with minimal samples poses a significant challenge. This requires a method that can both capture the broad diversity and the true characteristics of the target domain distribution. We present Conditional Relaxing Diffusion Inversion (CRDI), an innovative `training-free' approach designed to enhance distribution diversity in synthetic image generation. Distinct from conventional methods, CRDI does not rely on fine-tuning based on only a few samples. Instead, it focuses on reconstructing each target image instance and expanding diversity through few-shot learning. The approach initiates by identifying a Sample-wise Guidance Embedding (SGE) for the diffusion model, which serves a purpose analogous to the explicit latent codes in certain Generative Adversarial Network (GAN) models. Subsequently, the method involves a scheduler that progressively introduces perturbations to the SGE, thereby augmenting diversity. Comprehensive experiments demonstrates that our method surpasses GAN-based reconstruction techniques and equals state-of-the-art (SOTA) FSIG methods in performance. Additionally, it effectively mitigates overfitting and catastrophic forgetting, common drawbacks of fine-tuning approaches.