Multi-task Geometric Estimation of Depth and Surface Normal from Monocular 360{\deg} Images

TL;DR

This paper presents a multi-task learning network that simultaneously estimates depth and surface normals from 360-degree images, significantly improving accuracy and robustness over existing methods in complex scenes.

Contribution

The paper introduces a novel multi-task architecture with a fusion module for joint depth and surface normal estimation from panoramic images, enhancing scene understanding.

Findings

Outperforms state-of-the-art in both tasks

Handles intricate textures effectively

Establishes new benchmark in 360-degree geometric estimation

Abstract

Geometric estimation is required for scene understanding and analysis in panoramic 360{\deg} images. Current methods usually predict a single feature, such as depth or surface normal. These methods can lack robustness, especially when dealing with intricate textures or complex object surfaces. We introduce a novel multi-task learning (MTL) network that simultaneously estimates depth and surface normals from 360{\deg} images. Our first innovation is our MTL architecture, which enhances predictions for both tasks by integrating geometric information from depth and surface normal estimation, enabling a deeper understanding of 3D scene structure. Another innovation is our fusion module, which bridges the two tasks, allowing the network to learn shared representations that improve accuracy and robustness. Experimental results demonstrate that our MTL architecture significantly outperforms state-of-the-art methods in both depth and surface normal estimation, showing superior performance in complex and diverse scenes. Our model's effectiveness and generalizability, particularly in handling intricate surface textures, establish it as a new benchmark in 360{\deg} image geometric estimation. The code and model are available at \url{https://github.com/huangkun101230/360MTLGeometricEstimation}.