SAMPLING: Scene-adaptive Hierarchical Multiplane Images Representation for Novel View Synthesis from a Single Image

TL;DR

SAMPLING introduces a scene-adaptive hierarchical MPI approach for high-quality novel view synthesis from a single outdoor scene image, effectively handling unbounded scenes with improved geometric detail.

Contribution

It proposes an adaptive-bins MPI arrangement and hierarchical refinement to better model large-scale outdoor scenes from a single image.

Findings

Significant performance improvements on KITTI dataset.

Good generalization to unseen Tanks and Temples dataset.

Effective handling of unbounded outdoor scenes.

Abstract

Recent novel view synthesis methods obtain promising results for relatively small scenes, e.g., indoor environments and scenes with a few objects, but tend to fail for unbounded outdoor scenes with a single image as input. In this paper, we introduce SAMPLING, a Scene-adaptive Hierarchical Multiplane Images Representation for Novel View Synthesis from a Single Image based on improved multiplane images (MPI). Observing that depth distribution varies significantly for unbounded outdoor scenes, we employ an adaptive-bins strategy for MPI to arrange planes in accordance with each scene image. To represent intricate geometry and multi-scale details, we further introduce a hierarchical refinement branch, which results in high-quality synthesized novel views. Our method demonstrates considerable performance gains in synthesizing large-scale unbounded outdoor scenes using a single image on the KITTI dataset and generalizes well to the unseen Tanks and Temples dataset.The code and models will soon be made available.