Transformer-based Image Compression

TL;DR

This paper introduces a Transformer-based image compression method that leverages a VAE architecture with novel neural transformation units and attention modules, achieving competitive performance with fewer parameters.

Contribution

The paper presents a new Transformer-based image compression framework using NTUs and a casual attention module, reducing model size while maintaining high compression quality.

Findings

Outperforms state-of-the-art CNN-based LIC methods

Requires up to 45% fewer model parameters

Achieves comparable results to VVC intra profile

Abstract

A Transformer-based Image Compression (TIC) approach is developed which reuses the canonical variational autoencoder (VAE) architecture with paired main and hyper encoder-decoders. Both main and hyper encoders are comprised of a sequence of neural transformation units (NTUs) to analyse and aggregate important information for more compact representation of input image, while the decoders mirror the encoder-side operations to generate pixel-domain image reconstruction from the compressed bitstream. Each NTU is consist of a Swin Transformer Block (STB) and a convolutional layer (Conv) to best embed both long-range and short-range information; In the meantime, a casual attention module (CAM) is devised for adaptive context modeling of latent features to utilize both hyper and autoregressive priors. The TIC rivals with state-of-the-art approaches including deep convolutional neural networks (CNNs) based learnt image coding (LIC) methods and handcrafted rules-based intra profile of recently-approved Versatile Video Coding (VVC) standard, and requires much less model parameters, e.g., up to 45% reduction to leading-performance LIC.