Rethinking Bj{\o}ntegaard Delta for Compression Efficiency Evaluation: Are We Calculating It Precisely and Reliably?

TL;DR

This paper critically reevaluates the Bj{46}ntegaard Delta metric for codec performance comparison, proposing a neural network-based approach for more accurate and reliable R-D curve estimation to improve validity in compression research.

Contribution

It introduces a neural network method for precise R-D curve modeling, addressing estimation reliability issues in traditional BD calculations.

Findings

Neural network approach improves R-D curve estimation accuracy.

Traditional BD estimation may be unreliable with limited data points.

Recommendations for rigorous R-D sampling and reliability assessment.

Abstract

For decades, the Bj{\o}ntegaard Delta (BD) has been the metric for evaluating codec Rate-Distortion (R-D) performance. Yet, in most studies, BD is determined using just 4-5 R-D data points, could this be sufficient? As codecs and quality metrics advance, does the conventional BD estimation still hold up? Crucially, are the performance improvements of new codecs and tools genuine, or merely artifacts of estimation flaws? This paper addresses these concerns by reevaluating BD estimation. We present a novel approach employing a parameterized deep neural network to model R-D curves with high precision across various metrics, accompanied by a comprehensive R-D dataset. This approach both assesses the reliability of BD calculations and serves as a precise BD estimator. Our findings advocate for the adoption of rigorous R-D sampling and reliability metrics in future compression research to ensure the validity and reliability of results.