Event-enhanced Passive Non-line-of-sight imaging for moving objects with Physical embedding

TL;DR

This paper introduces a novel event-enhanced passive NLOS imaging prototype that uses bio-inspired sensors and neural networks to improve imaging of moving objects around corners, overcoming traditional ill-posed inverse problems.

Contribution

It presents the first sensor-dominated restoration prototype utilizing dynamic vision sensors for NLOS imaging of moving objects with physical embedding.

Findings

Event-based sensors outperform traditional sensors in NLOS imaging.

Simulation datasets effectively pre-train physical models for real-world application.

The prototype demonstrates superior accuracy in passive NLOS imaging of moving objects.

Abstract

Non-line-of-sight (NLOS) imaging with intelligent sensors emerges as a novel technique in imaging and sensing occluded objects around corners. With the innovation of bio-inspired neuromorphic sensors, the applications of novel sensors in unconventional imaging tasks like NLOS imaging have shown promising prospects in intelligent perception, encompassing autonomous driving, medical endoscopy and other sensing scenarios. However, the most challenging point of sensors application in computational imaging is the inverse problem established between sensors acquisition and reconstructions. Traditional physical retrieval methods with certain sensors applications usually result in poor reconstruction due to the highly ill-posedness, particularly in moving object imaging. Thanks to the development of neural networks, data-driven methods have greatly improved its accuracy, however, heavy reliance on data volume has put great pressure on data collection and dataset fabrication. To the best of our knowledge, we firstly propose a sensor-dominated restoration prototype termed "event enhanced passive NLOS imaging prototype for moving objects with physical embedding" (EPNP), which illustrated the application of dynamic vision sensors in NLOS imaging. EPNP induces an event camera for feature extraction of dynamic diffusion spot and leverages simulation dataset to pre-train the physical embedded model before fine-tuning with limited real-shot data. The proposed EPNP prototype is verified by simulation and real-world experiments, while the comparisons of data paradigms also validate the superiority of event-based sensor applications in passive NLOS imaging for moving objects and perspectives in advanced imaging techniques.