Inter-View Depth Consistency Testing in Depth Difference Subspace

TL;DR

This paper introduces a method for testing and improving depth consistency across multiple viewpoints in multiview depth imagery, enhancing virtual view synthesis quality for free-viewpoint television.

Contribution

It proposes a novel depth consistency testing approach in depth difference subspace and integrates it into a view synthesis algorithm to improve virtual view quality.

Findings

Objective quality of virtual views improved by up to 1.4 dB

Enhanced depth representation across multiple viewpoints

Improved subjective visual quality in free-viewpoint viewing

Abstract

Multiview depth imagery will play a critical role in free-viewpoint television. This technology requires high quality virtual view synthesis to enable viewers to move freely in a dynamic real world scene. Depth imagery at different viewpoints is used to synthesize an arbitrary number of novel views. Usually, depth images at multiple viewpoints are estimated individually by stereo-matching algorithms, and hence, show lack of interview consistency. This inconsistency affects the quality of view synthesis negatively. This paper proposes a method for depth consistency testing in depth difference subspace to enhance the depth representation of a scene across multiple viewpoints. Furthermore, we propose a view synthesis algorithm that uses the obtained consistency information to improve the visual quality of virtual views at arbitrary viewpoints. Our method helps us to find a linear subspace for our depth difference measurements in which we can test the inter-view consistency efficiently. With this, our approach is able to enhance the depth information for real world scenes. In combination with our consistency-adaptive view synthesis, we improve the visual experience of the free-viewpoint user. The experiments show that our approach enhances the objective quality of virtual views by up to 1.4 dB. The advantage for the subjective quality is also demonstrated.