NVC-1B: A Large Neural Video Coding Model

TL;DR

This paper introduces NVC-1B, the first billion-parameter neural video coding model, demonstrating significant performance improvements and setting a new state-of-the-art in video compression efficiency.

Contribution

The paper presents the design and development of NVC-1B, the first large-scale neural video coding model with over 1 billion parameters, and analyzes the impact of model size and architecture on compression performance.

Findings

NVC-1B outperforms smaller models in video compression efficiency.

Model architecture influences compression performance, with Transformer-based models showing advantages.

Scaling model size leads to significant improvements in video coding quality.

Abstract

The emerging large models have achieved notable progress in the fields of natural language processing and computer vision. However, large models for neural video coding are still unexplored. In this paper, we try to explore how to build a large neural video coding model. Based on a small baseline model, we gradually scale up the model sizes of its different coding parts, including the motion encoder-decoder, motion entropy model, contextual encoder-decoder, contextual entropy model, and temporal context mining module, and analyze the influence of model sizes on video compression performance. Then, we explore to use different architectures, including CNN, mixed CNN-Transformer, and Transformer architectures, to implement the neural video coding model and analyze the influence of model architectures on video compression performance. Based on our exploration results, we design the first neural video coding model with more than 1 billion parameters -- NVC-1B. Experimental results show that our proposed large model achieves a significant video compression performance improvement over the small baseline model, and represents the state-of-the-art compression efficiency. We anticipate large models may bring up the video coding technologies to the next level.