Humans as Light Bulbs: 3D Human Reconstruction from Thermal Reflection

TL;DR

This paper introduces a novel method for 3D human reconstruction using thermal reflections, enabling pose estimation even when the person is not visible to standard cameras, by modeling infrared reflections with a differentiable rendering approach.

Contribution

It presents an analysis-by-synthesis framework that jointly models objects, people, and thermal reflections for improved 3D human reconstruction from thermal data.

Findings

Effective in challenging scenarios with curved mirrors

Works when the person is completely unseen by visible cameras

Combines generative models with differentiable reflection rendering

Abstract

The relatively hot temperature of the human body causes people to turn into long-wave infrared light sources. Since this emitted light has a larger wavelength than visible light, many surfaces in typical scenes act as infrared mirrors with strong specular reflections. We exploit the thermal reflections of a person onto objects in order to locate their position and reconstruct their pose, even if they are not visible to a normal camera. We propose an analysis-by-synthesis framework that jointly models the objects, people, and their thermal reflections, which allows us to combine generative models with differentiable rendering of reflections. Quantitative and qualitative experiments show our approach works in highly challenging cases, such as with curved mirrors or when the person is completely unseen by a normal camera.