Joint Multi-scale Gated Transformer and Prior-guided Convolutional Network for Learned Image Compression

TL;DR

This paper introduces MGTPCN, a novel learned image compression framework combining multi-scale gated transformers and prior-guided convolutions, achieving superior compression performance and efficiency over existing methods.

Contribution

It proposes a joint architecture with a new prior-guided convolution and multi-scale gated transformer to enhance feature extraction in learned image compression.

Findings

Outperforms state-of-the-art algorithms in compression quality

Achieves better performance-complexity trade-off

Effectively extracts multi-scale and high-frequency features

Abstract

Recently, learned image compression methods have made remarkable achievements, some of which have outperformed the traditional image codec VVC. The advantages of learned image compression methods over traditional image codecs can be largely attributed to their powerful nonlinear transform coding. Convolutional layers and shifted window transformer (Swin-T) blocks are the basic units of neural networks, and their representation capabilities play an important role in nonlinear transform coding. In this paper, to improve the ability of the vanilla convolution to extract local features, we propose a novel prior-guided convolution (PGConv), where asymmetric convolutions (AConvs) and difference convolutions (DConvs) are introduced to strengthen skeleton elements and extract high-frequency information, respectively. A re-parameterization strategy is also used to reduce the computational complexity of PGConv. Moreover, to improve the ability of the Swin-T block to extract non-local features, we propose a novel multi-scale gated transformer (MGT), where dilated window-based multi-head self-attention blocks with different dilation rates and depth-wise convolution layers with different kernel sizes are used to extract multi-scale features, and a gate mechanism is introduced to enhance non-linearity. Finally, we propose a novel joint Multi-scale Gated Transformer and Prior-guided Convolutional Network (MGTPCN) for learned image compression. Experimental results show that our MGTPCN surpasses state-of-the-art algorithms with a better trade-off between performance and complexity.