SGFormer: Spherical Geometry Transformer for 360 Depth Estimation

TL;DR

SGFormer introduces a spherical geometry transformer that integrates spherical priors into depth estimation, effectively handling panoramic distortions and preserving spherical structures for improved 360 depth estimation.

Contribution

The paper proposes SGFormer, a novel spherical geometry transformer with a spherical prior decoder and specialized re-projection techniques, advancing 360 depth estimation accuracy.

Findings

Outperforms state-of-the-art methods on benchmark datasets.

Effectively preserves spherical structures during decoding.

Enhances global perception and depth structure sharpness.

Abstract

Panoramic distortion poses a significant challenge in 360 depth estimation, particularly pronounced at the north and south poles. Existing methods either adopt a bi-projection fusion strategy to remove distortions or model long-range dependencies to capture global structures, which can result in either unclear structure or insufficient local perception. In this paper, we propose a spherical geometry transformer, named SGFormer, to address the above issues, with an innovative step to integrate spherical geometric priors into vision transformers. To this end, we retarget the transformer decoder to a spherical prior decoder (termed SPDecoder), which endeavors to uphold the integrity of spherical structures during decoding. Concretely, we leverage bipolar re-projection, circular rotation, and curve local embedding to preserve the spherical characteristics of equidistortion, continuity, and surface distance, respectively. Furthermore, we present a query-based global conditional position embedding to compensate for spatial structure at varying resolutions. It not only boosts the global perception of spatial position but also sharpens the depth structure across different patches. Finally, we conduct extensive experiments on popular benchmarks, demonstrating our superiority over state-of-the-art solutions.