Extended monocular 3D imaging

TL;DR

This paper introduces EM3D, a novel monocular 3D imaging framework that leverages light's vectorial nature, enabling high-resolution, accurate 3D imaging of challenging scenes without prior data, and unlocking new material identification capabilities.

Contribution

The paper presents a compact monocular camera system that fuses diffraction and polarization cues for snapshot 3D imaging of difficult scenes, expanding the potential of minimal form factor vision systems.

Findings

Achieved million-pixel 3D point cloud for challenging scenes

Demonstrated accurate 3D imaging without data prior

Enabled material identification through combined depth and polarization data

Abstract

3D vision is of paramount importance for numerous applications ranging from machine intelligence to precision metrology. Despite much recent progress, the majority of 3D imaging hardware remains bulky and complicated and provides much lower image resolution compared to their 2D counterparts. Moreover, there are many well-known scenarios that existing 3D imaging solutions frequently fail. Here, we introduce an extended monocular 3D imaging (EM3D) framework that fully exploits the vectorial wave nature of light. Via the multi-stage fusion of diffraction- and polarization-based depth cues, using a compact monocular camera equipped with a diffractive-refractive hybrid lens, we experimentally demonstrate the snapshot acquisition of a million-pixel and accurate 3D point cloud for extended scenes that are traditionally challenging, including those with low texture, being highly reflective, or nearly transparent, without a data prior. Furthermore, we discover that the combination of depth and polarization information can unlock unique new opportunities in material identification, which may further expand machine intelligence for applications like target recognition and face anti-spoofing. The straightforward yet powerful architecture thus opens up a new path for a higher-dimensional machine vision in a minimal form factor, facilitating the deployment of monocular cameras for applications in much more diverse scenarios.