Exploring Long- and Short-Range Temporal Information for Learned Video Compression

TL;DR

This paper introduces a novel learned video compression method that leverages both long-range and short-range temporal information, using a temporal prior and progressive guided motion compensation to improve rate-distortion performance.

Contribution

It proposes a temporal prior for long-range information and a hierarchical, optical flow-guided motion compensation for short-range, enhancing learned video compression.

Findings

Outperforms state-of-the-art video compression methods in RD performance.

Effective exploitation of long- and short-range temporal information.

Code is publicly available for reproducibility.

Abstract

Learned video compression methods have gained a variety of interest in the video coding community since they have matched or even exceeded the rate-distortion (RD) performance of traditional video codecs. However, many current learning-based methods are dedicated to utilizing short-range temporal information, thus limiting their performance. In this paper, we focus on exploiting the unique characteristics of video content and further exploring temporal information to enhance compression performance. Specifically, for long-range temporal information exploitation, we propose temporal prior that can update continuously within the group of pictures (GOP) during inference. In that case temporal prior contains valuable temporal information of all decoded images within the current GOP. As for short-range temporal information, we propose a progressive guided motion compensation to achieve robust and effective compensation. In detail, we design a hierarchical structure to achieve multi-scale compensation. More importantly, we use optical flow guidance to generate pixel offsets between feature maps at each scale, and the compensation results at each scale will be used to guide the following scale's compensation. Sufficient experimental results demonstrate that our method can obtain better RD performance than state-of-the-art video compression approaches. The code is publicly available on: https://github.com/Huairui/LSTVC.