Referenceless Rate-Distortion Modeling with Learning from Bitstream and Pixel Features

TL;DR

This paper introduces a deep learning-based referenceless rate-distortion model that predicts bitrate accurately using only one-pass coding and features from bitstream and pixels, improving efficiency and precision.

Contribution

It proposes a novel hybrid referenceless feature-based R-QP modeling method that enhances bitrate prediction accuracy with minimal adjustments to modern codecs.

Findings

Reduces bitrate estimation error within 10% by 24.60% on average.

Uses a quadratic R-QP model derived from Cauchy distribution.

Significantly outperforms state-of-the-art methods in accuracy.

Abstract

Generally, adaptive bitrates for variable Internet bandwidths can be obtained through multi-pass coding. Referenceless prediction-based methods show practical benefits compared with multi-pass coding to avoid excessive computational resource consumption, especially in low-latency circumstances. However, most of them fail to predict precisely due to the complex inner structure of modern codecs. Therefore, to improve the fidelity of prediction, we propose a referenceless prediction-based R-QP modeling (PmR-QP) method to estimate bitrate by leveraging a deep learning algorithm with only one-pass coding. It refines the global rate-control paradigm in modern codecs on flexibility and applicability with few adjustments as possible. By exploring the potentials of bitstream and pixel features from the prerequisite of one-pass coding, it can reach the expectation of bitrate estimation in terms of precision. To be more specific, we first describe the R-QP relationship curve as a robust quadratic R-QP modeling function derived from the Cauchy-based distribution. Second, we simplify the modeling function by fastening one operational point of the relationship curve received from the coding process. Third, we learn the model parameters from bitstream and pixel features, named them hybrid referenceless features, comprising texture information, hierarchical coding structure, and selected modes in intra-prediction. Extensive experiments demonstrate the proposed method significantly decreases the proportion of samples' bitrate estimation error within 10% by 24.60% on average over the state-of-the-art.