Depth Fields: Extending Light Field Techniques to Time-of-Flight Imaging

TL;DR

This paper introduces depth fields, a hybrid imaging technique combining light field and time-of-flight methods to enhance depth sensing capabilities and address limitations of individual approaches.

Contribution

It proposes a novel on-chip hybrid depth sensor that integrates light field and TOF imaging, enabling advanced applications like virtual aperture synthesis and improved depth mapping.

Findings

Depth fields improve depth resolution and robustness in complex scenes.

Hybrid approach enables virtual aperture synthesis for enhanced imaging.

Effective phase unwrapping in single-frequency TOF using depth fields.

Abstract

A variety of techniques such as light field, structured illumination, and time-of-flight (TOF) are commonly used for depth acquisition in consumer imaging, robotics and many other applications. Unfortunately, each technique suffers from its individual limitations preventing robust depth sensing. In this paper, we explore the strengths and weaknesses of combining light field and time-of-flight imaging, particularly the feasibility of an on-chip implementation as a single hybrid depth sensor. We refer to this combination as depth field imaging. Depth fields combine light field advantages such as synthetic aperture refocusing with TOF imaging advantages such as high depth resolution and coded signal processing to resolve multipath interference. We show applications including synthesizing virtual apertures for TOF imaging, improved depth mapping through partial and scattering occluders, and single frequency TOF phase unwrapping. Utilizing space, angle, and temporal coding, depth fields can improve depth sensing in the wild and generate new insights into the dimensions of light's plenoptic function.