SphereFusion: Efficient Panorama Depth Estimation via Gated Fusion

TL;DR

SphereFusion is an end-to-end framework that combines multiple projection methods and a gated fusion module to efficiently estimate panorama depth with high accuracy and speed, overcoming challenges of distortion and texture loss.

Contribution

It introduces SphereFusion, a novel method that fuses features from different projections for improved panorama depth estimation, achieving state-of-the-art speed and competitive accuracy.

Findings

Achieves the fastest inference speed of 17 ms on 512×1024 images.

Demonstrates competitive depth estimation accuracy on three public datasets.

Effectively handles distortion and texture loss issues in panorama images.

Abstract

Due to the rapid development of panorama cameras, the task of estimating panorama depth has attracted significant attention from the computer vision community, especially in applications such as robot sensing and autonomous driving. However, existing methods relying on different projection formats often encounter challenges, either struggling with distortion and discontinuity in the case of equirectangular, cubemap, and tangent projections, or experiencing a loss of texture details with the spherical projection. To tackle these concerns, we present SphereFusion, an end-to-end framework that combines the strengths of various projection methods. Specifically, SphereFusion initially employs 2D image convolution and mesh operations to extract two distinct types of features from the panorama image in both equirectangular and spherical projection domains. These features are then projected onto the spherical domain, where a gate fusion module selects the most reliable features for fusion. Finally, SphereFusion estimates panorama depth within the spherical domain. Meanwhile, SphereFusion employs a cache strategy to improve the efficiency of mesh operation. Extensive experiments on three public panorama datasets demonstrate that SphereFusion achieves competitive results with other state-of-the-art methods, while presenting the fastest inference speed at only 17 ms on a 512$\times$1024 panorama image.