TD-Paint: Faster Diffusion Inpainting Through Time Aware Pixel Conditioning

TL;DR

TD-Paint introduces a time-aware diffusion inpainting method that accelerates sampling by effectively utilizing known pixel information from the start, improving efficiency without sacrificing quality.

Contribution

It proposes a novel time-aware diffusion process that models variable noise levels at pixel level, enabling faster inpainting without architectural changes.

Findings

Significantly faster sampling times compared to state-of-the-art models

Maintains high image quality in inpainting tasks

Reduces model complexity while improving efficiency

Abstract

Diffusion models have emerged as highly effective techniques for inpainting, however, they remain constrained by slow sampling rates. While recent advances have enhanced generation quality, they have also increased sampling time, thereby limiting scalability in real-world applications. We investigate the generative sampling process of diffusion-based inpainting models and observe that these models make minimal use of the input condition during the initial sampling steps. As a result, the sampling trajectory deviates from the data manifold, requiring complex synchronization mechanisms to realign the generation process. To address this, we propose Time-aware Diffusion Paint (TD-Paint), a novel approach that adapts the diffusion process by modeling variable noise levels at the pixel level. This technique allows the model to efficiently use known pixel values from the start, guiding the generation process toward the target manifold. By embedding this information early in the diffusion process, TD-Paint significantly accelerates sampling without compromising image quality. Unlike conventional diffusion-based inpainting models, which require a dedicated architecture or an expensive generation loop, TD-Paint achieves faster sampling times without architectural modifications. Experimental results across three datasets show that TD-Paint outperforms state-of-the-art diffusion models while maintaining lower complexity.