Humans as Light Bulbs: 3D Human Reconstruction from Thermal Reflection
Ruoshi Liu, Carl Vondrick

TL;DR
This paper introduces a novel method for 3D human reconstruction using thermal reflections, enabling pose estimation even when the person is not visible to standard cameras, by modeling infrared reflections with a differentiable rendering approach.
Contribution
It presents an analysis-by-synthesis framework that jointly models objects, people, and thermal reflections for improved 3D human reconstruction from thermal data.
Findings
Effective in challenging scenarios with curved mirrors
Works when the person is completely unseen by visible cameras
Combines generative models with differentiable reflection rendering
Abstract
The relatively hot temperature of the human body causes people to turn into long-wave infrared light sources. Since this emitted light has a larger wavelength than visible light, many surfaces in typical scenes act as infrared mirrors with strong specular reflections. We exploit the thermal reflections of a person onto objects in order to locate their position and reconstruct their pose, even if they are not visible to a normal camera. We propose an analysis-by-synthesis framework that jointly models the objects, people, and their thermal reflections, which allows us to combine generative models with differentiable rendering of reflections. Quantitative and qualitative experiments show our approach works in highly challenging cases, such as with curved mirrors or when the person is completely unseen by a normal camera.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsVideo Surveillance and Tracking Methods · Advanced Vision and Imaging · Human Pose and Action Recognition
