Conditional Residual Coding with Explicit-Implicit Temporal Buffering for Learned Video Compression

TL;DR

This paper introduces a hybrid explicit-implicit temporal buffering scheme for learned video compression, reducing memory requirements while maintaining high coding performance by combining explicit frame references with a small set of implicit features.

Contribution

It proposes a novel hybrid buffering strategy that outperforms single-type methods and significantly reduces memory needs in learned video coding.

Findings

Hybrid buffering outperforms single explicit or implicit methods.

Memory usage is reduced to the equivalent of two video frames.

Negligible performance drop on 2K videos with the new scheme.

Abstract

This work proposes a hybrid, explicit-implicit temporal buffering scheme for conditional residual video coding. Recent conditional coding methods propagate implicit temporal information for inter-frame coding, demonstrating superior coding performance to those relying exclusively on previously decoded frames (i.e. the explicit temporal information). However, these methods require substantial memory to store a large number of implicit features. This work presents a hybrid buffering strategy. For inter-frame coding, it buffers one previously decoded frame as the explicit temporal reference and a small number of learned features as implicit temporal reference. Our hybrid buffering scheme for conditional residual coding outperforms the single use of explicit or implicit information. Moreover, it allows the total buffer size to be reduced to the equivalent of two video frames with a negligible performance drop on 2K video sequences. The ablation experiment further sheds light on how these two types of temporal references impact the coding performance.