Geometric Transformation-Embedded Mamba for Learned Video Compression

TL;DR

This paper introduces a novel learned video compression framework that uses geometric transformations and a direct nonlinear transform strategy, outperforming existing methods in perceptual quality and temporal consistency at low bitrates.

Contribution

The work proposes a cascaded Mamba module with embedded geometric transformations and a locality refinement network, along with a conditional entropy model, to improve learned video compression.

Findings

Outperforms state-of-the-art methods in perceptual quality

Achieves better temporal consistency at low bitrates

Demonstrates effectiveness of geometric transformation embedding

Abstract

Although learned video compression methods have exhibited outstanding performance, most of them typically follow a hybrid coding paradigm that requires explicit motion estimation and compensation, resulting in a complex solution for video compression. In contrast, we introduce a streamlined yet effective video compression framework founded on a direct transform strategy, i.e., nonlinear transform, quantization, and entropy coding. We first develop a cascaded Mamba module (CMM) with different embedded geometric transformations to effectively explore both long-range spatial and temporal dependencies. To improve local spatial representation, we introduce a locality refinement feed-forward network (LRFFN) that incorporates a hybrid convolution block based on difference convolutions. We integrate the proposed CMM and LRFFN into the encoder and decoder of our compression framework. Moreover, we present a conditional channel-wise entropy model that effectively utilizes conditional temporal priors to accurately estimate the probability distributions of current latent features. Extensive experiments demonstrate that our method outperforms state-of-the-art video compression approaches in terms of perceptual quality and temporal consistency under low-bitrate constraints. Our source codes and models will be available at https://github.com/cshw2021/GTEM-LVC.