ShinyNeRF: Digitizing Anisotropic Appearance in Neural Radiance Fields

TL;DR

ShinyNeRF is a novel neural radiance field framework that accurately models anisotropic specular reflections, enabling high-fidelity 3D digitization and editing of complex material appearances.

Contribution

It introduces a method to jointly estimate surface normals, tangents, and anisotropic properties within NeRFs, handling both isotropic and anisotropic reflections for improved realism.

Findings

Achieves state-of-the-art performance in modeling anisotropic reflections.

Provides plausible physical interpretations of material properties.

Enables editing of material appearance in 3D digitizations.

Abstract

Recent advances in digitization technologies have transformed the preservation and dissemination of cultural heritage. In this vein, Neural Radiance Fields (NeRF) have emerged as a leading technology for 3D digitization, delivering representations with exceptional realism. However, existing methods struggle to accurately model anisotropic specular surfaces, typically observed, for example, on brushed metals. In this work, we introduce ShinyNeRF, a novel framework capable of handling both isotropic and anisotropic reflections. Our method is capable of jointly estimating surface normals, tangents, specular concentration, and anisotropy magnitudes of an Anisotropic Spherical Gaussian (ASG) distribution, by learning an approximation of the outgoing radiance as an encoded mixture of isotropic von Mises-Fisher (vMF) distributions. Experimental results show that ShinyNeRF not only achieves state-of-the-art performance on digitizing anisotropic specular reflections, but also offers plausible physical interpretations and editing of material properties compared to existing methods.