NeRF-Enhanced Outpainting for Faithful Field-of-View Extrapolation

TL;DR

This paper introduces NeRF-Enhanced Outpainting (NEO), a novel method that uses NeRF-generated extended field-of-view images to improve scene-faithful outpainting for applications like robotic navigation.

Contribution

The paper proposes a new problem of faithful FOV extrapolation and presents NEO, a simple yet effective solution leveraging NeRF for scene-specific outpainting training.

Findings

Demonstrates robustness on multiple datasets

Outperforms existing outpainting methods in scene fidelity

Provides a foundation for future research in FOV extrapolation

Abstract

In various applications, such as robotic navigation and remote visual assistance, expanding the field of view (FOV) of the camera proves beneficial for enhancing environmental perception. Unlike image outpainting techniques aimed solely at generating aesthetically pleasing visuals, these applications demand an extended view that faithfully represents the scene. To achieve this, we formulate a new problem of faithful FOV extrapolation that utilizes a set of pre-captured images as prior knowledge of the scene. To address this problem, we present a simple yet effective solution called NeRF-Enhanced Outpainting (NEO) that uses extended-FOV images generated through NeRF to train a scene-specific image outpainting model. To assess the performance of NEO, we conduct comprehensive evaluations on three photorealistic datasets and one real-world dataset. Extensive experiments on the benchmark datasets showcase the robustness and potential of our method in addressing this challenge. We believe our work lays a strong foundation for future exploration within the research community.