MomentsNeRF: Leveraging Orthogonal Moments for Few-Shot Neural Rendering

TL;DR

MomentsNeRF introduces a novel neural rendering framework that leverages orthogonal moments like Gabor and Zernike to improve 3D scene synthesis in few-shot settings, outperforming existing methods in quality and robustness.

Contribution

It is the first to incorporate orthogonal moments into NeRF, enabling effective transfer learning and scene reconstruction with minimal input images.

Findings

Achieves higher PSNR, SSIM, LPIPS, and DISTS metrics than previous methods.

Effectively synthesizes complex textures and shapes with reduced noise and artifacts.

Outperforms state-of-the-art in novel view synthesis and 3D reconstruction.

Abstract

We propose MomentsNeRF, a novel framework for one- and few-shot neural rendering that predicts a neural representation of a 3D scene using Orthogonal Moments. Our architecture offers a new transfer learning method to train on multi-scenes and incorporate a per-scene optimization using one or a few images at test time. Our approach is the first to successfully harness features extracted from Gabor and Zernike moments, seamlessly integrating them into the NeRF architecture. We show that MomentsNeRF performs better in synthesizing images with complex textures and shapes, achieving a significant noise reduction, artifact elimination, and completing the missing parts compared to the recent one- and few-shot neural rendering frameworks. Extensive experiments on the DTU and Shapenet datasets show that MomentsNeRF improves the state-of-the-art by {3.39\;dB\;PSNR}, 11.1% SSIM, 17.9% LPIPS, and 8.3% DISTS metrics. Moreover, it outperforms state-of-the-art performance for both novel view synthesis and single-image 3D view reconstruction. The source code is accessible at: https://amughrabi.github.io/momentsnerf/.