Video Compression with Hierarchical Temporal Neural Representation

TL;DR

This paper introduces TeNeRV, a hierarchical neural video representation that captures complex temporal dependencies through feature fusion and group-specific modulation, leading to improved compression performance over existing INR-based methods.

Contribution

We propose TeNeRV, a novel hierarchical neural network for video compression that models short- and long-term dependencies with adaptive group priors, enhancing rate-distortion efficiency.

Findings

TeNeRV outperforms existing INR-based methods in rate-distortion metrics.

The IFF module improves local temporal coherence and motion capture.

GAM enables adaptive representations across different Groups-of-Pictures.

Abstract

Video compression has recently benefited from implicit neural representations (INRs), which model videos as continuous functions. INRs offer compact storage and flexible reconstruction, providing a promising alternative to traditional codecs. However, most existing INR-based methods treat the temporal dimension as an independent input, limiting their ability to capture complex temporal dependencies. To address this, we propose a Hierarchical Temporal Neural Representation for Videos, TeNeRV. TeNeRV integrates short- and long-term dependencies through two key components. First, an Inter-Frame Feature Fusion (IFF) module aggregates features from adjacent frames, enforcing local temporal coherence and capturing fine-grained motion. Second, a GoP-Adaptive Modulation (GAM) mechanism partitions videos into Groups-of-Pictures and learns group-specific priors. The mechanism modulates network parameters, enabling adaptive representations across different GoPs. Extensive experiments demonstrate that TeNeRV consistently outperforms existing INR-based methods in rate-distortion performance, validating the effectiveness of our proposed approach.