Adaptive Rate Control for Deep Video Compression with Rate-Distortion Prediction

TL;DR

This paper introduces a neural network-based rate control scheme for deep video compression that predicts coding parameters directly from uncompressed frames, improving accuracy and stability without pre-encoding.

Contribution

It proposes a novel content-aware, neural network-driven rate control method that estimates rate-distortion relationships for each frame in deep video compression.

Findings

Achieves high rate control accuracy at mini-GOP level

Reduces inter-frame quality fluctuations

Operates with low computational overhead

Abstract

Deep video compression has made significant progress in recent years, achieving rate-distortion performance that surpasses that of traditional video compression methods. However, rate control schemes tailored for deep video compression have not been well studied. In this paper, we propose a neural network-based $\lambda$-domain rate control scheme for deep video compression, which determines the coding parameter $\lambda$ for each to-be-coded frame based on the rate-distortion-$\lambda$ (R-D-$\lambda$) relationships directly learned from uncompressed frames, achieving high rate control accuracy efficiently without the need for pre-encoding. Moreover, this content-aware scheme is able to mitigate inter-frame quality fluctuations and adapt to abrupt changes in video content. Specifically, we introduce two neural network-based predictors to estimate the relationship between bitrate and $\lambda$, as well as the relationship between distortion and $\lambda$ for each frame. Then we determine the coding parameter $\lambda$ for each frame to achieve the target bitrate. Experimental results demonstrate that our approach achieves high rate control accuracy at the mini-GOP level with low time overhead and mitigates inter-frame quality fluctuations across video content of varying resolutions.