NeurMiPs: Neural Mixture of Planar Experts for View Synthesis

TL;DR

NeurMiPs introduces a scene representation combining local planar experts and neural radiance fields, enabling efficient and high-quality novel view synthesis by leveraging ray-plane intersections and compositing techniques.

Contribution

The paper proposes NeurMiPs, a novel planar-based scene representation that blends explicit mesh rendering with neural radiance fields for improved view synthesis.

Findings

Superior performance over existing methods in view synthesis

Faster rendering speeds

Effective modeling of geometry and appearance

Abstract

We present Neural Mixtures of Planar Experts (NeurMiPs), a novel planar-based scene representation for modeling geometry and appearance. NeurMiPs leverages a collection of local planar experts in 3D space as the scene representation. Each planar expert consists of the parameters of the local rectangular shape representing geometry and a neural radiance field modeling the color and opacity. We render novel views by calculating ray-plane intersections and composite output colors and densities at intersected points to the image. NeurMiPs blends the efficiency of explicit mesh rendering and flexibility of the neural radiance field. Experiments demonstrate superior performance and speed of our proposed method, compared to other 3D representations in novel view synthesis.