Imaging Hidden Objects with Consumer LiDAR via Motion Induced Sampling

TL;DR

This paper demonstrates non-line-of-sight imaging using consumer-grade LiDAR on smartphones by introducing a motion-induced sampling model and a multi-frame fusion strategy, enabling 3D reconstruction and object tracking.

Contribution

The authors propose a novel motion-induced aperture sampling model and a multi-frame fusion method to achieve NLOS imaging on low-cost, off-the-shelf LiDAR sensors.

Findings

Achieved NLOS imaging capabilities on smartphone-grade LiDAR.

Enabled 3D reconstruction and object tracking of hidden objects.

Demonstrated plug-and-play NLOS imaging with minimal hardware and setup.

Abstract

LiDARs are being increasingly deployed for consumer imaging in handheld, wearable, and robotic applications. These sensors can capture the time-of-flight of light at picosecond resolution, which in principle, enables them to capture information about objects hidden from their field of view. While such non-line-of-sight (NLOS) imaging capabilities have been shown on research-grade LiDARs, they are challenging to achieve on consumer devices due to poor signal quality resulting from low laser power, low spatial resolution, and object and camera motion. Inspired by burst photography and synthetic aperture radar, we propose a multi-frame fusion strategy to overcome these challenges and demonstrate NLOS imaging on consumer LiDAR. We first introduce the motion-induced aperture sampling model to unify the effects of object shape, object motion, and camera motion under a single measurement model. Using this model, we demonstrate several NLOS capabilities on a smartphone-grade LiDAR: (1) 3D reconstruction, (2) single and multi-object tracking, and (3) camera localization using hidden objects. Previously, NLOS imaging capabilities were largely restricted to bulky and expensive research-grade hardware that requires extensive setup and calibration. Our results represent a shift towards plug-and-play NLOS imaging, where anyone can image hidden objects with off-the-shelf hardware ($<100) and no additional setup. We believe that democratization of such capabilities will advance consumer applications of NLOS imaging.