Enhancing AI Face Realism: Cost-Efficient Quality Improvement in Distilled Diffusion Models with a Fully Synthetic Dataset

TL;DR

This paper introduces a cost-effective method to improve the quality of images generated by distilled diffusion models using synthetic datasets and a refinement network, achieving near-baseline quality at significantly reduced computational costs.

Contribution

The study proposes a novel synthetic dataset-based training approach for enhancing distilled diffusion models, reducing computational costs while maintaining high image quality.

Findings

Achieved up to 82% reduction in computational cost.

Generated photorealistic portraits comparable to baseline models.

Demonstrated effectiveness of synthetic paired datasets for model refinement.

Abstract

This study presents a novel approach to enhance the cost-to-quality ratio of image generation with diffusion models. We hypothesize that differences between distilled (e.g. FLUX.1-schnell) and baseline (e.g. FLUX.1-dev) models are consistent and, therefore, learnable within a specialized domain, like portrait generation. We generate a synthetic paired dataset and train a fast image-to-image translation head. Using two sets of low- and high-quality synthetic images, our model is trained to refine the output of a distilled generator (e.g., FLUX.1-schnell) to a level comparable to a baseline model like FLUX.1-dev, which is more computationally intensive. Our results show that the pipeline, which combines a distilled version of a large generative model with our enhancement layer, delivers similar photorealistic portraits to the baseline version with up to an 82% decrease in computational cost compared to FLUX.1-dev. This study demonstrates the potential for improving the efficiency of AI solutions involving large-scale image generation.