Efficient Sub-pixel Motion Compensation in Learned Video Codecs

TL;DR

This paper enhances learned video codecs' sub-pixel motion compensation by integrating advanced interpolation filters and block-based motion, resulting in over 10% better compression and reduced decoding complexity.

Contribution

It introduces a novel approach to improve learned codecs' motion compensation inspired by conventional codecs, combining advanced filters and block-based motion.

Findings

Over 10% rate reduction in compression.

Decoding complexity reduced from 391 to 214 MAC per pixel.

Open-source implementation available.

Abstract

Motion compensation is a key component of video codecs. Conventional codecs (HEVC and VVC) have carefully refined this coding step, with an important focus on sub-pixel motion compensation. On the other hand, learned codecs achieve sub-pixel motion compensation through simple bilinear filtering. This paper offers to improve learned codec motion compensation by drawing inspiration from conventional codecs. It is shown that the usage of more advanced interpolation filters, block-based motion information and finite motion accuracy lead to better compression performance and lower decoding complexity. Experimental results are provided on the Cool-chic video codec, where we demonstrate a rate decrease of more than 10% and a lowering of motion-related decoding complexity from 391 MAC per pixel to 214 MAC per pixel. All contributions are made open-source at https://github.com/Orange-OpenSource/Cool-Chic