L-STEC: Learned Video Compression with Long-term Spatio-Temporal Enhanced Context

TL;DR

L-STEC introduces a novel neural video compression method that captures long-term dependencies and enhances spatio-temporal context, significantly improving compression efficiency and preserving fine details.

Contribution

The paper proposes L-STEC, combining LSTM-based long-term dependency modeling with warped spatial context fusion for improved neural video compression.

Findings

Achieves 37.01% bitrate savings in PSNR

Outperforms VTM-17.0 and DCVC-FM in benchmarks

Establishes new state-of-the-art performance

Abstract

Neural Video Compression has emerged in recent years, with condition-based frameworks outperforming traditional codecs. However, most existing methods rely solely on the previous frame's features to predict temporal context, leading to two critical issues. First, the short reference window misses long-term dependencies and fine texture details. Second, propagating only feature-level information accumulates errors over frames, causing prediction inaccuracies and loss of subtle textures. To address these, we propose the Long-term Spatio-Temporal Enhanced Context (L-STEC) method. We first extend the reference chain with LSTM to capture long-term dependencies. We then incorporate warped spatial context from the pixel domain, fusing spatio-temporal information through a multi-receptive field network to better preserve reference details. Experimental results show that L-STEC significantly improves compression by enriching contextual information, achieving 37.01% bitrate savings in PSNR and 31.65% in MS-SSIM compared to DCVC-TCM, outperforming both VTM-17.0 and DCVC-FM and establishing new state-of-the-art performance.