RePer-360: Releasing Perspective Priors for 360$^\circ$ Depth Estimation via Self-Modulation

TL;DR

RePer-360 introduces a novel distortion-aware self-modulation framework that effectively adapts depth foundation models for 360° panoramic depth estimation, outperforming standard fine-tuning with significantly less data.

Contribution

It proposes a lightweight geometry-aligned guidance module and Self-Conditioned AdaLN-Zero mechanism to adapt pretrained models to panoramic images without overwriting perspective priors.

Findings

Surpasses standard fine-tuning methods using only 1% of training data.

Achieves approximately 20% RMSE improvement in in-domain settings.

Enhances training stability and cross-projection alignment with cubemap-domain consistency loss.

Abstract

Recent depth foundation models trained on perspective imagery achieve strong performance, yet generalize poorly to 360$^\circ$ images due to the substantial geometric discrepancy between perspective and panoramic domains. Moreover, fully fine-tuning these models typically requires large amounts of panoramic data. To address this issue, we propose RePer-360, a distortion-aware self-modulation framework for monocular panoramic depth estimation that adapts depth foundation models while preserving powerful pretrained perspective priors. Specifically, we design a lightweight geometry-aligned guidance module to derive a modulation signal from two complementary projections (i.e., ERP and CP) and use it to guide the model toward the panoramic domain without overwriting its pretrained perspective knowledge. We further introduce a Self-Conditioned AdaLN-Zero mechanism that produces pixel-wise scaling factors to reduce the feature distribution gap between the perspective and panoramic domains. In addition, a cubemap-domain consistency loss further improves training stability and cross-projection alignment. By shifting the focus from complementary-projection fusion to panoramic domain adaptation under preserved pretrained perspective priors, RePer-360 surpasses standard fine-tuning methods while using only 1\% of the training data. Under the same in-domain training setting, it further achieves an approximately 20\% improvement in RMSE. Code will be released upon acceptance.