CoDe-NeRF: Neural Rendering via Dynamic Coefficient Decomposition

TL;DR

CoDe-NeRF introduces a dynamic coefficient decomposition framework that enhances neural rendering by better modeling view-dependent appearances, especially specular highlights, leading to sharper and more realistic scene synthesis.

Contribution

It proposes a novel decomposition approach with a shared neural basis and dynamic coefficients conditioned on view and illumination, improving rendering of complex appearances.

Findings

Produces sharper, more realistic specular highlights

Outperforms existing methods on challenging benchmarks

Enhances modeling of view-dependent appearance

Abstract

Neural Radiance Fields (NeRF) have shown impressive performance in novel view synthesis, but challenges remain in rendering scenes with complex specular reflections and highlights. Existing approaches may produce blurry reflections due to entanglement between lighting and material properties, or encounter optimization instability when relying on physically-based inverse rendering. In this work, we present a neural rendering framework based on dynamic coefficient decomposition, aiming to improve the modeling of view-dependent appearance. Our approach decomposes complex appearance into a shared, static neural basis that encodes intrinsic material properties, and a set of dynamic coefficients generated by a Coefficient Network conditioned on view and illumination. A Dynamic Radiance Integrator then combines these components to synthesize the final radiance. Experimental results on several challenging benchmarks suggest that our method can produce sharper and more realistic specular highlights compared to existing techniques. We hope that this decomposition paradigm can provide a flexible and effective direction for modeling complex appearance in neural scene representations.