Improving Deep Video Compression by Resolution-adaptive Flow Coding

TL;DR

This paper introduces Resolution-adaptive Flow Coding (RaFC), a novel framework that enhances deep video compression by adaptively selecting optimal resolutions for flow map encoding at both frame and block levels, improving efficiency.

Contribution

The paper proposes a new multi-resolution approach for flow map compression in deep video codecs, with adaptive resolution selection at frame and block levels based on rate-distortion criteria.

Findings

RaFC significantly improves compression efficiency on benchmark datasets.

Adaptive resolution selection outperforms fixed-resolution methods.

The framework achieves better rate-distortion performance across multiple datasets.

Abstract

In the learning based video compression approaches, it is an essential issue to compress pixel-level optical flow maps by developing new motion vector (MV) encoders. In this work, we propose a new framework called Resolution-adaptive Flow Coding (RaFC) to effectively compress the flow maps globally and locally, in which we use multi-resolution representations instead of single-resolution representations for both the input flow maps and the output motion features of the MV encoder. To handle complex or simple motion patterns globally, our frame-level scheme RaFC-frame automatically decides the optimal flow map resolution for each video frame. To cope different types of motion patterns locally, our block-level scheme called RaFC-block can also select the optimal resolution for each local block of motion features. In addition, the rate-distortion criterion is applied to both RaFC-frame and RaFC-block and select the optimal motion coding mode for effective flow coding. Comprehensive experiments on four benchmark datasets HEVC, VTL, UVG and MCL-JCV clearly demonstrate the effectiveness of our overall RaFC framework after combing RaFC-frame and RaFC-block for video compression.