DeepInv: A Novel Self-supervised Learning Approach for Fast and Accurate Diffusion Inversion

TL;DR

DeepInv introduces a self-supervised, trainable diffusion inversion method that significantly improves speed and accuracy without requiring ground-truth noise labels, enabling better controllable image editing.

Contribution

The paper presents the first trainable solver for diffusion inversion that predicts noise step-by-step using self-supervised learning and data augmentation.

Findings

Achieves +40.435% SSIM over EasyInv.

Runs 9887.5% faster than ReNoise.

Outperforms existing methods in both speed and accuracy.

Abstract

Diffusion inversion is a task of recovering the noise of an image in a diffusion model, which is vital for controllable diffusion image editing. At present, diffusion inversion still remains a challenging task due to the lack of viable supervision signals. Thus, most existing methods resort to approximation-based solutions, which however are often at the cost of performance or efficiency. To remedy these shortcomings, we propose a novel self-supervised diffusion inversion approach in this paper, termed Deep Inversion (DeepInv). Instead of requiring ground-truth noise annotations, we introduce a self-supervised objective as well as a data augmentation strategy to generate high-quality pseudo noises from real images without manual intervention. Based on these two innovative designs, DeepInv is also equipped with an iterative and multi-scale training regime to train a parameterized inversion solver, thereby achieving the fast and accurate image-to-noise mapping. To the best of our knowledge, this is the first attempt of presenting a trainable solver to predict inversion noise step by step. The extensive experiments show that our DeepInv can achieve much better performance and inference speed than the compared methods, e.g., +40.435% SSIM than EasyInv and +9887.5% speed than ReNoise on COCO dataset. Moreover, our careful designs of trainable solvers can also provide insights to the community. Codes and model parameters will be released in https://github.com/potato-kitty/DeepInv.