On-the-go Reflectance Transformation Imaging with Ordinary Smartphones

TL;DR

This paper introduces a mobile, smartphone-based RTI method that uses two devices and neural relighting to efficiently capture and reconstruct surface reflectance, enabling field use without specialized hardware.

Contribution

It presents a novel on-the-go RTI technique using ordinary smartphones and neural models, eliminating the need for dedicated hardware setups.

Findings

Outperforms state-of-the-art hardware-based RTI methods.

Enables real-time, field-based surface reflectance capture.

Uses neural relighting with PCA compression for efficient reconstruction.

Abstract

Reflectance Transformation Imaging (RTI) is a popular technique that allows the recovery of per-pixel reflectance information by capturing an object under different light conditions. This can be later used to reveal surface details and interactively relight the subject. Such process, however, typically requires dedicated hardware setups to recover the light direction from multiple locations, making the process tedious when performed outside the lab. We propose a novel RTI method that can be carried out by recording videos with two ordinary smartphones. The flash led-light of one device is used to illuminate the subject while the other captures the reflectance. Since the led is mounted close to the camera lenses, we can infer the light direction for thousands of images by freely moving the illuminating device while observing a fiducial marker surrounding the subject. To deal with such amount of data, we propose a neural relighting model that reconstructs object appearance for arbitrary light directions from extremely compact reflectance distribution data compressed via Principal Components Analysis (PCA). Experiments shows that the proposed technique can be easily performed on the field with a resulting RTI model that can outperform state-of-the-art approaches involving dedicated hardware setups.