Correcting the Sub-optimal Bit Allocation

TL;DR

This paper identifies and corrects a flaw in a recent bit allocation method for Neural Video Compression, leading to significant improvements in rate-distortion performance through a new, more accurate algorithm.

Contribution

The paper reveals the sub-optimality of a recent bit allocation approach and introduces a corrected, practical algorithm that significantly enhances compression efficiency.

Findings

Corrected bit allocation improves R-D performance.

Significant bitrate error reduction.

Outperforms existing methods by a large margin.

Abstract

In this paper, we investigate the problem of bit allocation in Neural Video Compression (NVC). First, we reveal that a recent bit allocation approach claimed to be optimal is, in fact, sub-optimal due to its implementation. Specifically, we find that its sub-optimality lies in the improper application of semi-amortized variational inference (SAVI) on latent with non-factorized variational posterior. Then, we show that the corrected version of SAVI on non-factorized latent requires recursively applying back-propagating through gradient ascent, based on which we derive the corrected optimal bit allocation algorithm. Due to the computational in-feasibility of the corrected bit allocation, we design an efficient approximation to make it practical. Empirical results show that our proposed correction significantly improves the incorrect bit allocation in terms of R-D performance and bitrate error, and outperforms all other bit allocation methods by a large margin. The source code is provided in the supplementary material.