Casual 6-DoF: free-viewpoint panorama using a handheld 360 camera

TL;DR

This paper introduces a novel approach for free-viewpoint 6-DoF panorama generation using a single handheld 360-degree camera, enabling quick data capture and high-quality scene reconstruction.

Contribution

It presents a new method combining 360-degree data capture, spherical depth estimation, and free-viewpoint rendering, improving efficiency and quality over traditional perspective-based approaches.

Findings

Outperforms state-of-the-art methods in visual quality

Demonstrates robustness across various environments

Enables faster 6-DoF scene reconstruction

Abstract

Six degrees-of-freedom (6-DoF) video provides telepresence by enabling users to move around in the captured scene with a wide field of regard. Compared to methods requiring sophisticated camera setups, the image-based rendering method based on photogrammetry can work with images captured with any poses, which is more suitable for casual users. However, existing image-based rendering methods are based on perspective images. When used to reconstruct 6-DoF views, it often requires capturing hundreds of images, making data capture a tedious and time-consuming process. In contrast to traditional perspective images, 360{\deg} images capture the entire surrounding view in a single shot, thus, providing a faster capturing process for 6-DoF view reconstruction. This paper presents a novel method to provide 6-DoF experiences over a wide area using an unstructured collection of 360{\deg} panoramas captured by a conventional 360{\deg} camera. Our method consists of 360{\deg} data capturing, novel depth estimation to produce a high-quality spherical depth panorama, and high-fidelity free-viewpoint generation. We compared our method against state-of-the-art methods, using data captured in various environments. Our method shows better visual quality and robustness in the tested scenes.