TeX-NeRF: Neural Radiance Fields from Pseudo-TeX Vision

TL;DR

TeX-NeRF is a novel 3D scene reconstruction method that uses infrared images and Pseudo-TeX vision to achieve high-quality results and accurate temperature estimations, especially useful in low-light conditions.

Contribution

This work introduces TeX-NeRF, a new infrared-based 3D reconstruction approach that incorporates object emissivity and Pseudo-TeX vision, filling a gap in scene reconstruction under challenging lighting conditions.

Findings

Achieves scene reconstruction quality comparable to RGB-based methods.

Provides accurate temperature estimations of scene objects.

Introduces the 3D-TeX datasets for infrared and Pseudo-TeX images.

Abstract

Neural radiance fields (NeRF) has gained significant attention for its exceptional visual effects. However, most existing NeRF methods reconstruct 3D scenes from RGB images captured by visible light cameras. In practical scenarios like darkness, low light, or bad weather, visible light cameras become ineffective. Therefore, we propose TeX-NeRF, a 3D reconstruction method using only infrared images, which introduces the object material emissivity as a priori, preprocesses the infrared images using Pseudo-TeX vision, and maps the temperatures (T), emissivities (e), and textures (X) of the scene into the saturation (S), hue (H), and value (V) channels of the HSV color space, respectively. Novel view synthesis using the processed images has yielded excellent results. Additionally, we introduce 3D-TeX Datasets, the first dataset comprising infrared images and their corresponding Pseudo-TeX vision images. Experiments demonstrate that our method not only matches the quality of scene reconstruction achieved with high-quality RGB images but also provides accurate temperature estimations for objects in the scene.