InfiniteNature-Zero: Learning Perpetual View Generation of Natural Scenes from Single Images

TL;DR

InfiniteNature-Zero is a self-supervised method that generates long, realistic view sequences of natural scenes from a single image, without needing multiple views or camera pose information during training.

Contribution

It introduces a novel self-supervised training paradigm for perpetual view generation from single images, eliminating the need for multi-view videos or pose data.

Findings

Outperforms supervised methods requiring multi-view videos.

Generates hundreds of diverse, realistic views from a single image.

Learns stable view synthesis without explicit camera pose supervision.

Abstract

We present a method for learning to generate unbounded flythrough videos of natural scenes starting from a single view, where this capability is learned from a collection of single photographs, without requiring camera poses or even multiple views of each scene. To achieve this, we propose a novel self-supervised view generation training paradigm, where we sample and rendering virtual camera trajectories, including cyclic ones, allowing our model to learn stable view generation from a collection of single views. At test time, despite never seeing a video during training, our approach can take a single image and generate long camera trajectories comprised of hundreds of new views with realistic and diverse content. We compare our approach with recent state-of-the-art supervised view generation methods that require posed multi-view videos and demonstrate superior performance and synthesis quality.