Fast and accurate neural reflectance transformation imaging through knowledge distillation

TL;DR

This paper introduces a knowledge distillation method to make neural reflectance transformation imaging faster and more practical for high-resolution applications, improving upon previous neural approaches.

Contribution

It presents a novel knowledge distillation technique to significantly reduce the computational cost of neural RTI, enabling real-time high-resolution relighting.

Findings

Achieves faster rendering times with minimal quality loss

Maintains high reflectance accuracy comparable to original neural RTI

Enables practical use of neural RTI on limited hardware

Abstract

Reflectance Transformation Imaging (RTI) is very popular for its ability to visually analyze surfaces by enhancing surface details through interactive relighting, starting from only a few tens of photographs taken with a fixed camera and variable illumination. Traditional methods like Polynomial Texture Maps (PTM) and Hemispherical Harmonics (HSH) are compact and fast, but struggle to accurately capture complex reflectance fields using few per-pixel coefficients and fixed bases, leading to artifacts, especially in highly reflective or shadowed areas. The NeuralRTI approach, which exploits a neural autoencoder to learn a compact function that better approximates the local reflectance as a function of light directions, has been shown to produce superior quality at comparable storage cost. However, as it performs interactive relighting with custom decoder networks with many parameters, the rendering step is computationally expensive and not feasible at full resolution for large images on limited hardware. Earlier attempts to reduce costs by directly training smaller networks have failed to produce valid results. For this reason, we propose to reduce its computational cost through a novel solution based on Knowledge Distillation (DisK-NeuralRTI). ...