ColorVideoVDP: A visual difference predictor for image, video and display distortions

TL;DR

ColorVideoVDP is a comprehensive video and image quality metric that models spatial and temporal visual perception for luminance and color, improving distortion prediction especially for AR/VR displays.

Contribution

It introduces a novel psychophysical model for chromatic contrast sensitivity and cross-channel masking, and a new XR-Display-Artifact-Video dataset for training and evaluation.

Findings

Significant improvement over existing metrics in predicting distortions.

Effective in assessing AR/VR display artifacts.

Applicable to various video streaming and display quality tasks.

Abstract

ColorVideoVDP is a video and image quality metric that models spatial and temporal aspects of vision, for both luminance and color. The metric is built on novel psychophysical models of chromatic spatiotemporal contrast sensitivity and cross-channel contrast masking. It accounts for the viewing conditions, geometric, and photometric characteristics of the display. It was trained to predict common video streaming distortions (e.g. video compression, rescaling, and transmission errors), and also 8 new distortion types related to AR/VR displays (e.g. light source and waveguide non-uniformities). To address the latter application, we collected our novel XR-Display-Artifact-Video quality dataset (XR-DAVID), comprised of 336 distorted videos. Extensive testing on XR-DAVID, as well as several datasets from the literature, indicate a significant gain in prediction performance compared to existing metrics. ColorVideoVDP opens the doors to many novel applications which require the joint automated spatiotemporal assessment of luminance and color distortions, including video streaming, display specification and design, visual comparison of results, and perceptually-guided quality optimization.