Bi-Directional Deep Contextual Video Compression

TL;DR

This paper introduces a bi-directional deep contextual video compression scheme for B-frames, significantly improving compression efficiency and outperforming traditional codecs like H.265/HEVC and even surpassing H.266/VVC in some cases.

Contribution

The paper presents a novel bi-directional deep contextual compression model with motion difference coding, multi-scale context utilization, and a hierarchical training strategy for improved B-frame video compression.

Findings

Achieves 26.6% BD-Rate reduction over H.265/HEVC

Outperforms H.266/VVC on certain datasets

Introduces effective bi-directional motion and context modeling

Abstract

Deep video compression has made remarkable process in recent years, with the majority of advancements concentrated on P-frame coding. Although efforts to enhance B-frame coding are ongoing, their compression performance is still far behind that of traditional bi-directional video codecs. In this paper, we introduce a bi-directional deep contextual video compression scheme tailored for B-frames, termed DCVC-B, to improve the compression performance of deep B-frame coding. Our scheme mainly has three key innovations. First, we develop a bi-directional motion difference context propagation method for effective motion difference coding, which significantly reduces the bit cost of bi-directional motions. Second, we propose a bi-directional contextual compression model and a corresponding bi-directional temporal entropy model, to make better use of the multi-scale temporal contexts. Third, we propose a hierarchical quality structure-based training strategy, leading to an effective bit allocation across large groups of pictures (GOP). Experimental results show that our DCVC-B achieves an average reduction of 26.6% in BD-Rate compared to the reference software for H.265/HEVC under random access conditions. Remarkably, it surpasses the performance of the H.266/VVC reference software on certain test datasets under the same configuration. We anticipate our work can provide valuable insights and bring up deep B-frame coding to the next level.