Hybrid Distortion Aggregated Visual Comfort Assessment for Stereoscopic Image Retargeting

TL;DR

This paper introduces HDA-VCA, a comprehensive assessment scheme for stereoscopic retargeted images that combines multiple distortion types to better predict visual comfort in 3D media.

Contribution

The paper proposes a novel hybrid distortion aggregation method for visual comfort assessment in stereoscopic images, integrating structural, informational, binocular, and semantic distortions.

Findings

HDA-VCA outperforms existing schemes on multiple stereoscopic image datasets.

The scheme effectively captures complex distortions affecting visual comfort.

Validation shows improved correlation with human subjective assessments.

Abstract

Visual comfort is a quite important factor in 3D media service. Few research efforts have been carried out in this area especially in case of 3D content retargeting which may introduce more complicated visual distortions. In this paper, we propose a Hybrid Distortion Aggregated Visual Comfort Assessment (HDA-VCA) scheme for stereoscopic retargeted images (SRI), considering aggregation of hybrid distortions including structure distortion, information loss, binocular incongruity and semantic distortion. Specifically, a Local-SSIM feature is proposed to reflect the local structural distortion of SRI, and information loss is represented by Dual Natural Scene Statistics (D-NSS) feature extracted from the binocular summation and difference channels. Regarding binocular incongruity, visual comfort zone, window violation, binocular rivalry, and accommodation-vergence conflict of human visual system (HVS) are evaluated. Finally, the semantic distortion is represented by the correlation distance of paired feature maps extracted from original stereoscopic image and its retargeted image by using trained deep neural network. We validate the effectiveness of HDA-VCA on published Stereoscopic Image Retargeting Database (SIRD) and two stereoscopic image databases IEEE-SA and NBU 3D-VCA. The results demonstrate HDA-VCA's superior performance in handling hybrid distortions compared to state-of-the-art VCA schemes.