APT: Improving Diffusion Models for High Resolution Image Generation with Adaptive Path Tracing

TL;DR

This paper introduces Adaptive Path Tracing (APT), a novel framework that enhances high-resolution image generation with diffusion models by addressing patch-based limitations, resulting in clearer images and faster sampling.

Contribution

APT combines statistical matching and scale-aware scheduling to improve patch-based diffusion methods for high-resolution images, reducing artifacts and increasing efficiency.

Findings

Produces more detailed high-resolution images

Enables faster sampling with minimal quality loss

Outperforms existing patch-based methods in clarity and speed

Abstract

Latent Diffusion Models (LDMs) are generally trained at fixed resolutions, limiting their capability when scaling up to high-resolution images. While training-based approaches address this limitation by training on high-resolution datasets, they require large amounts of data and considerable computational resources, making them less practical. Consequently, training-free methods, particularly patch-based approaches, have become a popular alternative. These methods divide an image into patches and fuse the denoising paths of each patch, showing strong performance on high-resolution generation. However, we observe two critical issues for patch-based approaches, which we call ``patch-level distribution shift" and ``increased patch monotonicity." To address these issues, we propose Adaptive Path Tracing (APT), a framework that combines Statistical Matching to ensure patch distributions remain consistent in upsampled latents and Scale-aware Scheduling to deal with the patch monotonicity. As a result, APT produces clearer and more refined details in high-resolution images. In addition, APT enables a shortcut denoising process, resulting in faster sampling with minimal quality degradation. Our experimental results confirm that APT produces more detailed outputs with improved inference speed, providing a practical approach to high-resolution image generation.