Adaptive Multiplane Image Generation from a Single Internet Picture

TL;DR

This paper introduces an efficient method for generating multiplane images from a single high-resolution photo, enabling novel view synthesis with significantly reduced computational costs and high-quality results.

Contribution

It proposes an adaptive MPI representation with variable image planes and lightweight CNNs for depth estimation and inpainting, improving efficiency over prior methods.

Findings

Produces high-quality novel views from a single image

Uses an order of magnitude fewer parameters than previous approaches

Demonstrates robustness on challenging internet images

Abstract

In the last few years, several works have tackled the problem of novel view synthesis from stereo images or even from a single picture. However, previous methods are computationally expensive, specially for high-resolution images. In this paper, we address the problem of generating a multiplane image (MPI) from a single high-resolution picture. We present the adaptive-MPI representation, which allows rendering novel views with low computational requirements. To this end, we propose an adaptive slicing algorithm that produces an MPI with a variable number of image planes. We present a new lightweight CNN for depth estimation, which is learned by knowledge distillation from a larger network. Occluded regions in the adaptive-MPI are inpainted also by a lightweight CNN. We show that our method is capable of producing high-quality predictions with one order of magnitude less parameters compared to previous approaches. The robustness of our method is evidenced on challenging pictures from the Internet.