Faithful Extreme Image Rescaling with Learnable Reversible Transformation and Semantic Priors

TL;DR

FaithEIR is a diffusion-based framework for extreme image rescaling that uses learnable reversible transformations and semantic priors to improve reconstruction fidelity and perceptual quality.

Contribution

It introduces a novel invertible transformation and adaptive detail prior to enhance extreme image rescaling performance.

Findings

Outperforms state-of-the-art methods in fidelity and perceptual quality.

Achieves superior results on extreme 16x rescaling tasks.

Provides a lightweight semantic embedder for improved conditioning.

Abstract

Most recent extreme rescaling methods struggle to preserve semantically consistent structures and produce realistic details, due to the severely ill-posed nature of low- to high-resolution mapping under scaling factors of $16\times$ or higher. To alleviate the above problems, we propose FaithEIR, a diffusion-based framework for extreme image rescaling. Inspired by singular value decomposition, we develop learnable reversible transformation that enables invertible downscaling and upscaling in the latent space. To compensate for information loss due to quantization, we propose an adaptive detail prior, a high-frequency dictionary that captures the empirical average of commonly occurring structures in the training data. Finally, we design a lightweight pixel semantic embedder to provide semantic conditioning for the pretrained diffusion model. We present extensive experimental results demonstrating that our FaithEIR consistently outperforms state-of-the-art methods, achieving superior reconstruction fidelity and perceptual quality. Our code, model weights, and detailed results are released at https://github.com/cshw2021/FaithEIR.