Region-Adaptive Generative Compression with Spatially Varying Diffusion Models

TL;DR

This paper introduces a region-adaptive diffusion image codec that allocates bits based on visual importance, enhancing perceptual quality and outperforming existing methods.

Contribution

It proposes a novel spatially varying diffusion model and integrates importance maps into the entropy model for improved rate-distortion performance.

Findings

Outperforms state-of-the-art ROI-controllable baselines in perceptual quality.

Supports non-uniform bit allocation within images.

Uses importance maps as priors to improve compression efficiency.

Abstract

Generative image codecs aim to optimize perceptual quality, producing realistic and detailed reconstructions. However, they often overlook a key property of human vision: our tendency to focus on particular aspects of a visual scene (e.g., salient objects) while giving less importance to other regions. An ideal perceptual codec should be able to exploit this property by allocating more representational capacity to perceptually important areas. To this end, we propose a region-adaptive diffusion-based image codec that supports non-uniform bit allocation within an image. We design a novel spatially varying diffusion model capable of denoising varying amounts of noise per pixel according to arbitrary importance maps. We further identify that these maps can serve as effective priors on the latent representation, and integrate them into our entropy model, improving rate-distortion performance. Built on these contributions, our spatially-adaptive diffusion-based codec outperforms state-of-the-art ROI-controllable baselines in both full-image and ROI-masked perceptual quality.