A Real-time 3D Desktop Display

TL;DR

This paper introduces an extended altiro3D library that processes real-time 2D video streams using AI to generate light-fields for 3D displays, enabling realistic 3D experiences from standard images or videos.

Contribution

The paper presents a novel real-time 3D desktop display system leveraging AI-enhanced light-field synthesis from 2D streams, with a user-friendly multi-platform GUI.

Findings

Real-time 3D rendering from 2D video streams achieved.

AI-based depth extraction improves 3D quality.

Supports standard desktop images and videos for 3D display.

Abstract

A new extended version of the altiro3D C++ Library -- initially developed to get glass-free holographic displays starting from 2D images -- is here introduced aiming to deal with 3D video streams from either 2D webcam images or flat video files. These streams are processed in real-time to synthesize light-fields (in Native format) and feed realistic 3D experiences. The core function needed to recreate multiviews consists on the use of MiDaS Convolutional Neural Network (CNN), which allows to extract a depth map from a single 2D image. Artificial Intelligence (AI) computing techniques are applied to improve the overall performance of the extended altiro3D Library. Thus, altiro3D can now treat standard images, video streams or screen portions of a Desktop where other apps may be also running (like web browsers, video chats, etc) and render them into 3D. To achieve the latter, a screen region need to be selected in order to feed the output directly into a light-field 3D device such as Looking Glass (LG) Portrait. In order to simplify the acquisition of a Desktop screen area by the user, a multi-platform Graphical User Interface has been also implemented. Sources available at: https://github.com/canessae/altiro3D/releases/tag/2.0.0