ResDiT: Evoking the Intrinsic Resolution Scalability in Diffusion Transformers

TL;DR

ResDiT is a training-free method that improves high-resolution image synthesis with diffusion transformers by rectifying positional encodings and enhancing local details, avoiding complex pipelines.

Contribution

It introduces a novel PE scaling technique and local-enhancement mechanism to address layout collapse and detail degradation in HR image generation.

Findings

ResDiT achieves high-fidelity HR image synthesis.

It effectively prevents spatial layout collapse.

The method integrates seamlessly with downstream tasks.

Abstract

Leveraging pre-trained Diffusion Transformers (DiTs) for high-resolution (HR) image synthesis often leads to spatial layout collapse and degraded texture fidelity. Prior work mitigates these issues with complex pipelines that first perform a base-resolution (i.e., training-resolution) denoising process to guide HR generation. We instead explore the intrinsic generative mechanisms of DiTs and propose ResDiT, a training-free method that scales resolution efficiently. We identify the core factor governing spatial layout, position embeddings (PEs), and show that the original PEs encode incorrect positional information when extrapolated to HR, which triggers layout collapse. To address this, we introduce a PE scaling technique that rectifies positional encoding under resolution changes. To further remedy low-fidelity details, we develop a local-enhancement mechanism grounded in base-resolution local attention. We design a patch-level fusion module that aggregates global and local cues, together with a Gaussian-weighted splicing strategy that eliminates grid artifacts. Comprehensive evaluations demonstrate that ResDiT consistently delivers high-fidelity, high-resolution image synthesis and integrates seamlessly with downstream tasks, including spatially controlled generation.