Hybrid Local-Global Context Learning for Neural Video Compression

TL;DR

This paper introduces a hybrid local-global context learning approach for neural video compression that combines flow-guided deformable compensation and flow-based warping to improve accuracy and reduce bit cost.

Contribution

It proposes a novel hybrid context generation module and a local-global context enhancement technique, improving motion compensation efficiency in neural video codecs.

Findings

Significant performance improvement over state-of-the-art methods

Effective reduction in bit cost for motion coding

Enhanced accuracy in complex scene motion estimation

Abstract

In neural video codecs, current state-of-the-art methods typically adopt multi-scale motion compensation to handle diverse motions. These methods estimate and compress either optical flow or deformable offsets to reduce inter-frame redundancy. However, flow-based methods often suffer from inaccurate motion estimation in complicated scenes. Deformable convolution-based methods are more robust but have a higher bit cost for motion coding. In this paper, we propose a hybrid context generation module, which combines the advantages of the above methods in an optimal way and achieves accurate compensation at a low bit cost. Specifically, considering the characteristics of features at different scales, we adopt flow-guided deformable compensation at largest-scale to produce accurate alignment in detailed regions. For smaller-scale features, we perform flow-based warping to save the bit cost for motion coding. Furthermore, we design a local-global context enhancement module to fully explore the local-global information of previous reconstructed signals. Experimental results demonstrate that our proposed Hybrid Local-Global Context learning (HLGC) method can significantly enhance the state-of-the-art methods on standard test datasets.