SAP: Segment Any 4K Panorama

TL;DR

SAP introduces a novel approach for high-resolution 4K panoramic instance segmentation by reformulating the task as perspective video segmentation along a spherical path, enabling effective zero-shot generalization.

Contribution

The paper presents a new panoramic segmentation model, SAP, trained on synthesized data, that outperforms existing models on real-world 4K panoramas by reformulating the problem as perspective video segmentation.

Findings

Achieves +17.2 zero-shot mIoU gain over vanilla SAM2 on 4K panorama benchmark.

Synthesizes 183,440 panoramic images with labels for large-scale training.

Effectively generalizes to real-world 360° images with high resolution.

Abstract

Promptable instance segmentation is widely adopted in embodied and AR systems, yet the performance of foundation models trained on perspective imagery often degrades on 360{\deg} panoramas. In this paper, we introduce Segment Any 4K Panorama (SAP), a foundation model for 4K high-resolution panoramic instance-level segmentation. We reformulate panoramic segmentation as fixed-trajectory perspective video segmentation, decomposing a panorama into overlapping perspective patches sampled along a continuous spherical traversal. This memory-aligned reformulation preserves native 4K resolution while restoring the smooth viewpoint transitions required for stable cross-view propagation. To enable large-scale supervision, we synthesize 183,440 4K-resolution panoramic images with instance segmentation labels using the InfiniGen engine. Trained under this trajectory-aligned paradigm, SAP generalizes effectively to real-world 360{\deg} images, achieving +17.2 zero-shot mIoU gain over vanilla SAM2 of different sizes on real-world 4K panorama benchmark.