ICME 2025 Generalizable HDR and SDR Video Quality Measurement Grand Challenge

TL;DR

The ICME 2025 Grand Challenge aims to develop and benchmark robust video quality assessment methods capable of accurately evaluating both HDR and SDR videos across diverse content and distortion types.

Contribution

This paper introduces a new benchmark challenge for generalizable HDR and SDR video quality measurement, with top models surpassing existing standards and setting new performance benchmarks.

Findings

Four methods outperformed VMAF baseline.

The top model achieved state-of-the-art performance.

The challenge promotes development of versatile VQA models.

Abstract

This paper reports IEEE International Conference on Multimedia \& Expo (ICME) 2025 Grand Challenge on Generalizable HDR and SDR Video Quality Measurement. With the rapid development of video technology, especially High Dynamic Range (HDR) and Standard Dynamic Range (SDR) contents, the need for robust and generalizable Video Quality Assessment (VQA) methods has become increasingly demanded. Existing VQA models often struggle to deliver consistent performance across varying dynamic ranges, distortion types, and diverse content. This challenge was established to benchmark and promote VQA approaches capable of jointly handling HDR and SDR content. In the final evaluation phase, five teams submitted seven models along with technical reports to the Full Reference (FR) and No Reference (NR) tracks. Among them, four methods outperformed VMAF baseline, while the top-performing model achieved state-of-the-art performance, setting a new benchmark for generalizable video quality assessment.