AligNeRF: High-Fidelity Neural Radiance Fields via Alignment-Aware Training

TL;DR

AligNeRF introduces an alignment-aware training method for high-resolution Neural Radiance Fields, effectively capturing detailed 3D scene representations while addressing data misalignment and reducing model complexity.

Contribution

It proposes a novel training strategy combining convolutional layers, alignment correction, and high-frequency loss to improve high-resolution NeRF reconstructions.

Findings

Better high-frequency detail recovery compared to state-of-the-art NeRFs.

Effective handling of data misalignment in dynamic scenes.

Reduced model complexity with convolutional layers.

Abstract

Neural Radiance Fields (NeRFs) are a powerful representation for modeling a 3D scene as a continuous function. Though NeRF is able to render complex 3D scenes with view-dependent effects, few efforts have been devoted to exploring its limits in a high-resolution setting. Specifically, existing NeRF-based methods face several limitations when reconstructing high-resolution real scenes, including a very large number of parameters, misaligned input data, and overly smooth details. In this work, we conduct the first pilot study on training NeRF with high-resolution data and propose the corresponding solutions: 1) marrying the multilayer perceptron (MLP) with convolutional layers which can encode more neighborhood information while reducing the total number of parameters; 2) a novel training strategy to address misalignment caused by moving objects or small camera calibration errors; and 3) a high-frequency aware loss. Our approach is nearly free without introducing obvious training/testing costs, while experiments on different datasets demonstrate that it can recover more high-frequency details compared with the current state-of-the-art NeRF models. Project page: \url{https://yifanjiang.net/alignerf.}