Cross-Platform Neural Video Coding: A Case Study

TL;DR

This paper addresses the mismatch issue in learning-based video codecs caused by floating-point errors, proposing static quantization to improve cross-platform compatibility and compression efficiency.

Contribution

It introduces a static quantization method for the hyper prior decoding path, enhancing cross-platform robustness of neural video codecs.

Findings

Mitigates encoder-decoder mismatch across different machines.

Reduces BD-rate increase to under 10% on benchmark sequences.

Improves compression efficiency without severe quality loss.

Abstract

In this paper, we first show that current learning-based video codecs, specifically the SSF codec, are not suitable for real-world applications due to the mismatch between the encoder and decoder caused by floating-point round-off errors. To address this issue, we propose the static quantization of the hyper prior decoding path. The quantization parameters are determined through an exhaustive search of all possible combinations of observers and quantization schemes from PyTorch. For the SSF codec, when encoding and decoding on different machines, the proposed solution effectively mitigates the mismatch issue and enhances compression efficiency results by preventing severe image quality degradation. When encoding and decoding are performed on the same machine, it constrains the average BD-rate increase to 9.93% and 9.02% for UVG and HEVC-B sequences, respectively.