NAS-NeRF: Generative Neural Architecture Search for Neural Radiance Fields

TL;DR

NAS-NeRF introduces a neural architecture search method that creates compact, scene-specific NeRF models, significantly reducing computational costs while maintaining high synthesis quality, thus enhancing deployability.

Contribution

The paper presents a generative neural architecture search strategy for NeRFs that produces scene-tailored, efficient architectures balancing complexity and quality, outperforming baseline models.

Findings

Achieves up to 5.74× smaller architectures

Reduces FLOPs by up to 4.19×

Speeds up rendering by up to 1.93×

Abstract

Neural radiance fields (NeRFs) enable high-quality novel view synthesis, but their high computational complexity limits deployability. While existing neural-based solutions strive for efficiency, they use one-size-fits-all architectures regardless of scene complexity. The same architecture may be unnecessarily large for simple scenes but insufficient for complex ones. Thus, there is a need to dynamically optimize the neural network component of NeRFs to achieve a balance between computational complexity and specific targets for synthesis quality. We introduce NAS-NeRF, a generative neural architecture search strategy that generates compact, scene-specialized NeRF architectures by balancing architecture complexity and target synthesis quality metrics. Our method incorporates constraints on target metrics and budgets to guide the search towards architectures tailored for each scene. Experiments on the Blender synthetic dataset show the proposed NAS-NeRF can generate architectures up to 5.74$\times$ smaller, with 4.19$\times$ fewer FLOPs, and 1.93$\times$ faster on a GPU than baseline NeRFs, without suffering a drop in SSIM. Furthermore, we illustrate that NAS-NeRF can also achieve architectures up to 23$\times$ smaller, with 22$\times$ fewer FLOPs, and 4.7$\times$ faster than baseline NeRFs with only a 5.3% average SSIM drop. Our source code is also made publicly available at https://saeejithnair.github.io/NAS-NeRF.