NeRF-SR: High-Quality Neural Radiance Fields using Supersampling

TL;DR

NeRF-SR enhances neural radiance fields for high-resolution view synthesis from low-res inputs by using supersampling and a refinement network that leverages depth and a single high-res reference image.

Contribution

The paper introduces NeRF-SR, a novel approach combining supersampling and depth-guided refinement to achieve high-quality HR view synthesis from LR images.

Findings

NeRF-SR produces high-resolution novel views with improved detail.

The method outperforms baseline NeRF in quality on synthetic and real datasets.

No external data is required for high-resolution synthesis.

Abstract

We present NeRF-SR, a solution for high-resolution (HR) novel view synthesis with mostly low-resolution (LR) inputs. Our method is built upon Neural Radiance Fields (NeRF) that predicts per-point density and color with a multi-layer perceptron. While producing images at arbitrary scales, NeRF struggles with resolutions that go beyond observed images. Our key insight is that NeRF benefits from 3D consistency, which means an observed pixel absorbs information from nearby views. We first exploit it by a supersampling strategy that shoots multiple rays at each image pixel, which further enforces multi-view constraint at a sub-pixel level. Then, we show that NeRF-SR can further boost the performance of supersampling by a refinement network that leverages the estimated depth at hand to hallucinate details from related patches on only one HR reference image. Experiment results demonstrate that NeRF-SR generates high-quality results for novel view synthesis at HR on both synthetic and real-world datasets without any external information.