What You Can Learn by Staring at a Blank Wall

TL;DR

This paper introduces a passive method that infers the presence and activity of people behind walls by analyzing subtle changes in indirect light on a wall, using neural networks trained on diverse scenes, achieving high accuracy without prior calibration.

Contribution

The method uniquely infers human presence and activity behind walls without known occluders or controlled lighting, generalizing across unknown rooms with no re-calibration.

Findings

Achieves approximately 94% accuracy in classifying number of people and activity.

Generalizes well to unseen environments and real-time settings.

Robust to scene variations and tested with real and synthetic data.

Abstract

We present a passive non-line-of-sight method that infers the number of people or activity of a person from the observation of a blank wall in an unknown room. Our technique analyzes complex imperceptible changes in indirect illumination in a video of the wall to reveal a signal that is correlated with motion in the hidden part of a scene. We use this signal to classify between zero, one, or two moving people, or the activity of a person in the hidden scene. We train two convolutional neural networks using data collected from 20 different scenes, and achieve an accuracy of $\approx94\%$ for both tasks in unseen test environments and real-time online settings. Unlike other passive non-line-of-sight methods, the technique does not rely on known occluders or controllable light sources, and generalizes to unknown rooms with no re-calibration. We analyze the generalization and robustness of our method with both real and synthetic data, and study the effect of the scene parameters on the signal quality.