EGIC: Enhanced Low-Bit-Rate Generative Image Compression Guided by Semantic Segmentation

TL;DR

EGIC is a novel image compression method that combines semantic segmentation guidance and residual control to achieve high-quality, low-bit-rate reconstructions with flexible distortion-perception trade-offs.

Contribution

It introduces two new components, OASIS-C and Output Residual Prediction, to enhance generative image compression and control over synthesis quality.

Findings

Outperforms state-of-the-art diffusion and GAN-based methods.

Nearly matches VTM-20.0 performance on distortion metrics.

Offers lightweight implementation with excellent interpolation.

Abstract

We introduce EGIC, an enhanced generative image compression method that allows traversing the distortion-perception curve efficiently from a single model. EGIC is based on two novel building blocks: i) OASIS-C, a conditional pre-trained semantic segmentation-guided discriminator, which provides both spatially and semantically-aware gradient feedback to the generator, conditioned on the latent image distribution, and ii) Output Residual Prediction (ORP), a retrofit solution for multi-realism image compression that allows control over the synthesis process by adjusting the impact of the residual between an MSE-optimized and GAN-optimized decoder output on the GAN-based reconstruction. Together, EGIC forms a powerful codec, outperforming state-of-the-art diffusion and GAN-based methods (e.g., HiFiC, MS-ILLM, and DIRAC-100), while performing almost on par with VTM-20.0 on the distortion end. EGIC is simple to implement, very lightweight, and provides excellent interpolation characteristics, which makes it a promising candidate for practical applications targeting the low bit range.