Content-Adaptive Rate-Quality Curve Prediction Model in Media Processing System

TL;DR

This paper introduces a flexible content-adaptive rate-quality prediction model for video encoding that improves efficiency and quality control in streaming services, with successful deployment in a real app.

Contribution

The proposed model predicts comprehensive bitrate-quality curves using codec, content, and anchor features, enabling flexible encoding adjustments without retraining.

Findings

Achieves 99.14% accuracy in VMAF quality prediction within 1 point.

Online A/B tests show +0.107% in video views and completions.

Model deployed on Xiaohongshu App demonstrates practical effectiveness.

Abstract

In streaming media services, video transcoding is a common practice to alleviate bandwidth demands. Unfortunately, traditional methods employing a uniform rate factor (RF) across all videos often result in significant inefficiencies. Content-adaptive encoding (CAE) techniques address this by dynamically adjusting encoding parameters based on video content characteristics. However, existing CAE methods are often tightly coupled with specific encoding strategies, leading to inflexibility. In this paper, we propose a model that predicts both RF-quality and RF-bitrate curves, which can be utilized to derive a comprehensive bitrate-quality curve. This approach facilitates flexible adjustments to the encoding strategy without necessitating model retraining. The model leverages codec features, content features, and anchor features to predict the bitrate-quality curve accurately. Additionally, we introduce an anchor suspension method to enhance prediction accuracy. Experiments confirm that the actual quality metric (VMAF) of the compressed video stays within 1 of the target, achieving an accuracy of 99.14%. By incorporating our quality improvement strategy with the rate-quality curve prediction model, we conducted online A/B tests, obtaining both +0.107% improvements in video views and video completions and +0.064% app duration time. Our model has been deployed on the Xiaohongshu App.