Deep3D: Fully Automatic 2D-to-3D Video Conversion with Deep Convolutional Neural Networks

TL;DR

Deep3D introduces an end-to-end deep learning approach for automatic 2D-to-3D video conversion, leveraging stereo pairs for improved performance without requiring ground truth depth maps.

Contribution

It presents a novel end-to-end neural network trained on stereo pairs, enabling more effective 2D-to-3D conversion compared to previous methods.

Findings

Deep3D outperforms baseline algorithms in quantitative metrics.

Human evaluations favor Deep3D's 3D conversions.

The approach leverages large datasets for training.

Abstract

As 3D movie viewing becomes mainstream and Virtual Reality (VR) market emerges, the demand for 3D contents is growing rapidly. Producing 3D videos, however, remains challenging. In this paper we propose to use deep neural networks for automatically converting 2D videos and images to stereoscopic 3D format. In contrast to previous automatic 2D-to-3D conversion algorithms, which have separate stages and need ground truth depth map as supervision, our approach is trained end-to-end directly on stereo pairs extracted from 3D movies. This novel training scheme makes it possible to exploit orders of magnitude more data and significantly increases performance. Indeed, Deep3D outperforms baselines in both quantitative and human subject evaluations.