Visual Comfort Aware-Reinforcement Learning for Depth Adjustment of Stereoscopic 3D Images

TL;DR

This paper introduces VCA-RL, a deep reinforcement learning method that models human-like decision making to optimize depth adjustment in stereoscopic 3D images, enhancing visual comfort and perception.

Contribution

The paper presents a novel DRL-based approach that explicitly models human sequential decision making for depth adjustment in stereoscopic images.

Findings

VCA-RL outperforms baseline methods in depth adjustment quality.

User studies confirm improved visual comfort with VCA-RL.

Effective across multiple S3D databases.

Abstract

Depth adjustment aims to enhance the visual experience of stereoscopic 3D (S3D) images, which accompanied with improving visual comfort and depth perception. For a human expert, the depth adjustment procedure is a sequence of iterative decision making. The human expert iteratively adjusts the depth until he is satisfied with the both levels of visual comfort and the perceived depth. In this work, we present a novel deep reinforcement learning (DRL)-based approach for depth adjustment named VCA-RL (Visual Comfort Aware Reinforcement Learning) to explicitly model human sequential decision making in depth editing operations. We formulate the depth adjustment process as a Markov decision process where actions are defined as camera movement operations to control the distance between the left and right cameras. Our agent is trained based on the guidance of an objective visual comfort assessment metric to learn the optimal sequence of camera movement actions in terms of perceptual aspects in stereoscopic viewing. With extensive experiments and user studies, we show the effectiveness of our VCA-RL model on three different S3D databases.