# Learned Rate Control for Frame-Level Adaptive Neural Video Compression via Dynamic Neural Network

## TL;DR

This paper introduces a dynamic neural video compression framework with rate control that adapts to target bitrates at runtime, improving efficiency and performance over existing methods.

## Contribution

It proposes the Dynamic-Route Autoencoder with a Rate Control Agent and a joint optimization strategy for variable bitrate neural video compression.

## Key findings

- Achieves 14.8% BD-Rate reduction over state-of-the-art methods.
- Gains 0.47dB BD-PSNR compared to existing approaches.
- Maintains 1.66% average bitrate error across experiments.

## Abstract

Neural Video Compression (NVC) has achieved remarkable performance in recent years. However, precise rate control remains a challenge due to the inherent limitations of learning-based codecs. To solve this issue, we propose a dynamic video compression framework designed for variable bitrate scenarios. First, to achieve variable bitrate implementation, we propose the Dynamic-Route Autoencoder with variable coding routes, each occupying partial computational complexity of the whole network and navigating to a distinct RD trade-off. Second, to approach the target bitrate, the Rate Control Agent estimates the bitrate of each route and adjusts the coding route of DRA at run time. To encompass a broad spectrum of variable bitrates while preserving overall RD performance, we employ the Joint-Routes Optimization strategy, achieving collaborative training of various routes. Extensive experiments on the HEVC and UVG datasets show that the proposed method achieves an average BD-Rate reduction of 14.8% and BD-PSNR gain of 0.47dB over state-of-the-art methods while maintaining an average bitrate error of 1.66%, achieving Rate-Distortion-Complexity Optimization (RDCO) for various bitrate and bitrate-constrained applications. Our code is available at https://git.openi.org.cn/OpenAICoding/DynamicDVC.