LatentINDIGO: An INN-Guided Latent Diffusion Algorithm for Image Restoration

TL;DR

LatentINDIGO introduces an INN-guided latent diffusion method for image restoration that effectively handles complex, unknown degradations while reducing computational costs, achieving state-of-the-art results.

Contribution

The paper proposes a wavelet-inspired invertible neural network integrated into latent diffusion models to improve robustness and efficiency in image restoration tasks.

Findings

Achieves state-of-the-art performance on synthetic and real-world images.

Handles complex and unknown degradations effectively.

Reduces computational and memory overhead compared to existing methods.

Abstract

There is a growing interest in the use of latent diffusion models (LDMs) for image restoration (IR) tasks due to their ability to model effectively the distribution of natural images. While significant progress has been made, there are still key challenges that need to be addressed. First, many approaches depend on a predefined degradation operator, making them ill-suited for complex or unknown degradations that deviate from standard analytical models. Second, many methods struggle to provide a stable guidance in the latent space and finally most methods convert latent representations back to the pixel domain for guidance at every sampling iteration, which significantly increases computational and memory overhead. To overcome these limitations, we introduce a wavelet-inspired invertible neural network (INN) that simulates degradations through a forward transform and reconstructs lost details via the inverse transform. We further integrate this design into a latent diffusion pipeline through two proposed approaches: LatentINDIGO-PixelINN, which operates in the pixel domain, and LatentINDIGO-LatentINN, which stays fully in the latent space to reduce complexity. Both approaches alternate between updating intermediate latent variables under the guidance of our INN and refining the INN forward model to handle unknown degradations. In addition, a regularization step preserves the proximity of latent variables to the natural image manifold. Experiments demonstrate that our algorithm achieves state-of-the-art performance on synthetic and real-world low-quality images, and can be readily adapted to arbitrary output sizes.