UniFuse: Unidirectional Fusion for 360$^{\circ}$ Panorama Depth Estimation

TL;DR

UniFuse introduces a unidirectional feature fusion method combining equirectangular and cubemap projections for panorama depth estimation, achieving state-of-the-art results with improved efficiency and generalization.

Contribution

The paper proposes a novel unidirectional fusion framework and an improved fusion module for panorama depth estimation, outperforming bidirectional methods in efficiency and accuracy.

Findings

Achieves state-of-the-art performance on four datasets.

More efficient than bidirectional fusion approaches.

Demonstrates better generalization and lower model complexity.

Abstract

Learning depth from spherical panoramas is becoming a popular research topic because a panorama has a full field-of-view of the environment and provides a relatively complete description of a scene. However, applying well-studied CNNs for perspective images to the standard representation of spherical panoramas, i.e., the equirectangular projection, is suboptimal, as it becomes distorted towards the poles. Another representation is the cubemap projection, which is distortion-free but discontinued on edges and limited in the field-of-view. This paper introduces a new framework to fuse features from the two projections, unidirectionally feeding the cubemap features to the equirectangular features only at the decoding stage. Unlike the recent bidirectional fusion approach operating at both the encoding and decoding stages, our fusion scheme is much more efficient. Besides, we also designed a more effective fusion module for our fusion scheme. Experiments verify the effectiveness of our proposed fusion strategy and module, and our model achieves state-of-the-art performance on four popular datasets. Additional experiments show that our model also has the advantages of model complexity and generalization capability.The code is available at https://github.com/alibaba/UniFuse-Unidirectional-Fusion.