High-Efficiency Neural Video Compression via Hierarchical Predictive Learning

TL;DR

This paper introduces DHVC 2.0, a neural video codec that uses hierarchical predictive coding to achieve high compression efficiency, real-time processing, and reduced memory usage without traditional motion estimation.

Contribution

The paper presents a novel hierarchical predictive coding framework for neural video compression that eliminates motion estimation, enabling faster processing and better scalability.

Findings

Outperforms existing methods in compression efficiency.

Supports real-time encoding and decoding on standard GPUs.

Enables progressive decoding suitable for network transmission.

Abstract

The enhanced Deep Hierarchical Video Compression-DHVC 2.0-has been introduced. This single-model neural video codec operates across a broad range of bitrates, delivering not only superior compression performance to representative methods but also impressive complexity efficiency, enabling real-time processing with a significantly smaller memory footprint on standard GPUs. These remarkable advancements stem from the use of hierarchical predictive coding. Each video frame is uniformly transformed into multiscale representations through hierarchical variational autoencoders. For a specific scale's feature representation of a frame, its corresponding latent residual variables are generated by referencing lower-scale spatial features from the same frame and then conditionally entropy-encoded using a probabilistic model whose parameters are predicted using same-scale temporal reference from previous frames and lower-scale spatial reference of the current frame. This feature-space processing operates from the lowest to the highest scale of each frame, completely eliminating the need for the complexity-intensive motion estimation and compensation techniques that have been standard in video codecs for decades. The hierarchical approach facilitates parallel processing, accelerating both encoding and decoding, and supports transmission-friendly progressive decoding, making it particularly advantageous for networked video applications in the presence of packet loss. Source codes will be made available.