Generalized Octave Convolutions for Learned Multi-Frequency Image Compression

TL;DR

This paper introduces a multi-frequency image compression method using octave convolutions to separate high and low frequency components, reducing redundancy and improving rate-distortion performance over existing learned and standard codecs.

Contribution

It presents the first learned multi-frequency compression approach based on octave convolutions, with novel architectures that better preserve spatial structure and outperform current methods.

Findings

Outperforms all existing learned methods and standard codecs on Kodak dataset.

Reduces spatial redundancy by factorizing latents into high and low frequency components.

Improves performance in other vision tasks like segmentation and denoising.

Abstract

Learned image compression has recently shown the potential to outperform the standard codecs. State-of-the-art rate-distortion (R-D) performance has been achieved by context-adaptive entropy coding approaches in which hyperprior and autoregressive models are jointly utilized to effectively capture the spatial dependencies in the latent representations. However, the latents are feature maps of the same spatial resolution in previous works, which contain some redundancies that affect the R-D performance. In this paper, we propose the first learned multi-frequency image compression and entropy coding approach that is based on the recently developed octave convolutions to factorize the latents into high and low frequency (resolution) components, where the low frequency is represented by a lower resolution. Therefore, its spatial redundancy is reduced, which improves the R-D performance. Novel generalized octave convolution and octave transposed-convolution architectures with internal activation layers are also proposed to preserve more spatial structure of the information. Experimental results show that the proposed scheme not only outperforms all existing learned methods as well as standard codecs such as the next-generation video coding standard VVC (4:2:0) on the Kodak dataset in both PSNR and MS-SSIM. We also show that the proposed generalized octave convolution can improve the performance of other auto-encoder-based computer vision tasks such as semantic segmentation and image denoising.