ArchSym: Detecting 3D-Grounded Architectural Symmetries in the Wild

TL;DR

This paper introduces a novel framework for detecting 3D-grounded architectural symmetries in real-world images, utilizing a new large-scale dataset and a symmetry detection model that outperforms existing methods.

Contribution

The paper presents the first method for 3D-grounded symmetry detection in in-the-wild images, including a scalable annotation pipeline and a single-view symmetry detector.

Findings

The dataset ArchSym enables effective training and evaluation.

The proposed detector significantly outperforms existing baselines.

The annotation pipeline is validated against geometry-based methods.

Abstract

Symmetry detection is a fundamental problem in computer vision, and symmetries serve as powerful priors for downstream tasks. However, existing learning-based methods for detecting 3D symmetries from single images have been almost exclusively trained and evaluated on object-centric or synthetic datasets, and thus fail to generalize to real-world scenes. Furthermore, due to the inherent scale ambiguity of monocular inputs, which makes localizing the 3D plane an ill-posed problem, many existing works only predict the plane's orientation. In this paper, we address these limitations by presenting the first framework for detecting 3D-grounded reflectional symmetries from single, in-the-wild RGB images, focusing on architectural landmarks. We introduce two key innovations: (1) a scalable data annotation pipeline to automatically curate a large-scale dataset of architectural symmetries, ArchSym, from SfM reconstructions by leveraging cross-view image matching; and building on the dataset, (2) a single-view symmetry detector that accurately localizes symmetries in 3D by parameterizing them as signed distance maps defined relative to predicted scene geometry. We validate our symmetry annotation pipeline against geometry-based alternatives and demonstrate that our symmetry detector significantly outperforms state-of-the-art baselines on our new benchmark.