Generating on Generated: An Approach Towards Self-Evolving Diffusion Models

TL;DR

This paper introduces a self-evolving approach for diffusion models using recursive self-improvement techniques to address training collapse caused by synthetic data, through perceptual alignment, preference sampling, and distribution-based weighting.

Contribution

It presents a novel framework applying recursive self-improvement to diffusion models, with strategies to mitigate hallucinations and improve training stability.

Findings

Enhanced perceptual alignment in generated data

Reduced hallucinations through preference sampling

Improved training stability and quality

Abstract

Recursive Self-Improvement (RSI) enables intelligence systems to autonomously refine their capabilities. This paper explores the application of RSI in text-to-image diffusion models, addressing the challenge of training collapse caused by synthetic data. We identify two key factors contributing to this collapse: the lack of perceptual alignment and the accumulation of generative hallucinations. To mitigate these issues, we propose three strategies: (1) a prompt construction and filtering pipeline designed to facilitate the generation of perceptual aligned data, (2) a preference sampling method to identify human-preferred samples and filter out generative hallucinations, and (3) a distribution-based weighting scheme to penalize selected samples with hallucinatory errors. Our extensive experiments validate the effectiveness of these approaches.