Improved Encoding for Overfitted Video Codecs

TL;DR

This paper introduces an improved neural video codec that uses optical flow guidance and joint rate-distortion optimization to enhance compression efficiency while maintaining low decoding complexity, approaching traditional codecs like HEVC.

Contribution

It proposes a novel method combining optical flow guidance and joint rate-distortion optimization for overfitted neural video codecs, achieving better compression with low complexity.

Findings

Compression performance close to HEVC

Decoding complexity of 1300 multiplications per pixel

Outperforms other overfitted codecs

Abstract

Overfitted neural video codecs offer a decoding complexity orders of magnitude smaller than their autoencoder counterparts. Yet, this low complexity comes at the cost of limited compression efficiency, in part due to their difficulty capturing accurate motion information. This paper proposes to guide motion information learning with an optical flow estimator. A joint rate-distortion optimization is also introduced to improve rate distribution across the different frames. These contributions maintain a low decoding complexity of 1300 multiplications per pixel while offering compression performance close to the conventional codec HEVC and outperforming other overfitted codecs. This work is made open-source at https://orange-opensource.github.io/Cool-Chic/