SE360: Semantic Edit in 360$^\circ$ Panoramas via Hierarchical Data Construction
Haoyi Zhong, Fang-Lue Zhang, Andrew Chalmers, Taehyun Rhee

TL;DR
SE360 introduces a hierarchical data construction framework for semantic editing in 360° panoramas, enabling realistic, guided object modifications with improved quality and accuracy over existing methods.
Contribution
The paper presents a novel coarse-to-fine data generation pipeline and a Transformer-based diffusion model for semantic editing in 360° panoramas, addressing previous limitations.
Findings
Outperforms existing methods in visual quality
Achieves higher semantic accuracy in edits
Ensures geometrically consistent data generation
Abstract
While instruction-based image editing is emerging, extending it to 360 panoramas introduces additional challenges. Existing methods often produce implausible results in both equirectangular projections (ERP) and perspective views. To address these limitations, we propose SE360, a novel framework for multi-condition guided object editing in 360 panoramas. At its core is a novel coarse-to-fine autonomous data generation pipeline without manual intervention. This pipeline leverages a Vision-Language Model (VLM) and adaptive projection adjustment for hierarchical analysis, ensuring the holistic segmentation of objects and their physical context. The resulting data pairs are both semantically meaningful and geometrically consistent, even when sourced from unlabeled panoramas. Furthermore, we introduce a cost-effective, two-stage data refinement strategy to improve data…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGenerative Adversarial Networks and Image Synthesis · Multimodal Machine Learning Applications · Computer Graphics and Visualization Techniques
