Deep Predictive Video Compression with Bi-directional Prediction

TL;DR

This paper introduces BP-DVC Net, a deep learning-based bi-predictive video compression network that leverages optical flow and bi-prediction to improve coding efficiency without transmitting motion vectors.

Contribution

The paper presents a novel deep learning framework for video compression that incorporates bi-prediction and optical flow estimation, eliminating the need for transmitting motion vectors.

Findings

Achieves compression performance comparable to H.264 and HEVC in PSNR and MS-SSIM.

Uses optical flow networks to improve coding efficiency without transmitting motion information.

Employs a bi-prediction network to minimize prediction residues.

Abstract

Recently, deep image compression has shown a big progress in terms of coding efficiency and image quality improvement. However, relatively less attention has been put on video compression using deep learning networks. In the paper, we first propose a deep learning based bi-predictive coding network, called BP-DVC Net, for video compression. Learned from the lesson of the conventional video coding, a B-frame coding structure is incorporated in our BP-DVC Net. While the bi-predictive coding in the conventional video codecs requires to transmit to decoder sides the motion vectors for block motion and the residues from prediction, our BP-DVC Net incorporates optical flow estimation networks in both encoder and decoder sides so as not to transmit the motion information to the decoder sides for coding efficiency improvement. Also, a bi-prediction network in the BP-DVC Net is proposed and used to precisely predict the current frame and to yield the resulting residues as small as possible. Furthermore, our BP-DVC Net allows for the compressive feature maps to be entropy-coded using the temporal context among the feature maps of adjacent frames. The BP-DVC Net has an end-to-end video compression architecture with newly designed flow and prediction losses. Experimental results show that the compression performance of our proposed method is comparable to those of H.264, HEVC in terms of PSNR and MS-SSIM.