Navigating Image Restoration with VAR's Distribution Alignment Prior

TL;DR

This paper introduces VarFormer, a novel image restoration framework leveraging VAR's distribution alignment prior, which improves generalization and efficiency in restoring images across multiple tasks.

Contribution

It proposes a new multi-scale latent representation approach within VAR, enhancing image restoration by leveraging distribution alignment priors for better generalization and reduced training costs.

Findings

Outperforms existing multi-task restoration methods

Achieves remarkable generalization on unseen tasks

Reduces training computational costs

Abstract

Generative models trained on extensive high-quality datasets effectively capture the structural and statistical properties of clean images, rendering them powerful priors for transforming degraded features into clean ones in image restoration. VAR, a novel image generative paradigm, surpasses diffusion models in generation quality by applying a next-scale prediction approach. It progressively captures both global structures and fine-grained details through the autoregressive process, consistent with the multi-scale restoration principle widely acknowledged in the restoration community. Furthermore, we observe that during the image reconstruction process utilizing VAR, scale predictions automatically modulate the input, facilitating the alignment of representations at subsequent scales with the distribution of clean images. To harness VAR's adaptive distribution alignment capability in image restoration tasks, we formulate the multi-scale latent representations within VAR as the restoration prior, thus advancing our delicately designed VarFormer framework. The strategic application of these priors enables our VarFormer to achieve remarkable generalization on unseen tasks while also reducing training computational costs. Extensive experiments underscores that our VarFormer outperforms existing multi-task image restoration methods across various restoration tasks.