Neural Rate Control for Video Encoding using Imitation Learning

TL;DR

This paper introduces a neural rate control policy for video encoding, formulated as a POMDP and learned via imitation learning, achieving improved bitrate efficiency in VP9 encoding.

Contribution

It presents a novel neural network-based rate control method learned through imitation from optimal trajectories, outperforming traditional policies in VP9 encoding.

Findings

Achieves 8.5% median bitrate reduction on real-world videos

Demonstrates improved encoding efficiency over libvpx

Utilizes imitation learning with auxiliary losses and data augmentation

Abstract

In modern video encoders, rate control is a critical component and has been heavily engineered. It decides how many bits to spend to encode each frame, in order to optimize the rate-distortion trade-off over all video frames. This is a challenging constrained planning problem because of the complex dependency among decisions for different video frames and the bitrate constraint defined at the end of the episode. We formulate the rate control problem as a Partially Observable Markov Decision Process (POMDP), and apply imitation learning to learn a neural rate control policy. We demonstrate that by learning from optimal video encoding trajectories obtained through evolution strategies, our learned policy achieves better encoding efficiency and has minimal constraint violation. In addition to imitating the optimal actions, we find that additional auxiliary losses, data augmentation/refinement and inference-time policy improvements are critical for learning a good rate control policy. We evaluate the learned policy against the rate control policy in libvpx, a widely adopted open source VP9 codec library, in the two-pass variable bitrate (VBR) mode. We show that over a diverse set of real-world videos, our learned policy achieves 8.5% median bitrate reduction without sacrificing video quality.