360MonoDepth: High-Resolution 360{\deg} Monocular Depth Estimation

TL;DR

This paper introduces 360MonoDepth, a novel framework that enables high-resolution 360-degree monocular depth estimation by projecting images onto tangent planes and recombining depth estimates for detailed, globally consistent 360-degree depth maps.

Contribution

It proposes a flexible tangent image-based approach for high-resolution 360-degree depth estimation, overcoming GPU memory limitations of existing CNN methods.

Findings

Produces dense, high-resolution 360-degree depth maps with detailed outdoor scene reconstruction.

Supports resolutions beyond 2K, suitable for VR and novel-view synthesis.

Achieves globally consistent disparity estimates through multi-scale alignment and blending.

Abstract

360{\deg} cameras can capture complete environments in a single shot, which makes 360{\deg} imagery alluring in many computer vision tasks. However, monocular depth estimation remains a challenge for 360{\deg} data, particularly for high resolutions like 2K (2048x1024) and beyond that are important for novel-view synthesis and virtual reality applications. Current CNN-based methods do not support such high resolutions due to limited GPU memory. In this work, we propose a flexible framework for monocular depth estimation from high-resolution 360{\deg} images using tangent images. We project the 360{\deg} input image onto a set of tangent planes that produce perspective views, which are suitable for the latest, most accurate state-of-the-art perspective monocular depth estimators. To achieve globally consistent disparity estimates, we recombine the individual depth estimates using deformable multi-scale alignment followed by gradient-domain blending. The result is a dense, high-resolution 360{\deg} depth map with a high level of detail, also for outdoor scenes which are not supported by existing methods. Our source code and data are available at https://manurare.github.io/360monodepth/.