Beyond Bj{\o}ntegaard: Limits of Video Compression Performance Comparisons

TL;DR

This paper critically examines the limitations of the Bj{\

Contribution

It introduces a new Akima interpolation method that improves the accuracy of performance metric comparisons in video codec evaluation.

Findings

Akima interpolation outperforms traditional methods in accuracy

Approximation error is below 1.5% for various metrics

Validates the use of advanced interpolation for diverse performance metrics

Abstract

For 20 years, the gold standard to evaluate the performance of video codecs is to calculate average differences between ratedistortion curves, also called the "Bj{\o}ntegaard Delta". With the help of this tool, the compression performance of codecs can be compared. In the past years, we could observe that the calculus was also deployed for other metrics than bitrate and distortion in terms of peak signal-to-noise ratio, for example other quality metrics such as video multi-method assessment fusion or hardware-dependent metrics such as the decoding energy. However, it is unclear whether the Bj{\o}ntegaard Delta is a valid way to evaluate these metrics. To this end, this paper reviews several interpolation methods and evaluates their accuracy using different performancemetrics. As a result, we propose to use a novel approach based on Akima interpolation, which returns the most accurate results for a large variety of performance metrics. The approximation accuracy of this new method is determined to be below a bound of 1.5%.