Beyond Planar Symmetry: Modeling human perception of reflection and rotation symmetries in the wild

TL;DR

This paper introduces Sym-NET, a deep learning model trained on human-labeled data to accurately detect reflection and rotation symmetries in images, outperforming existing algorithms and handling complex symmetry scenarios.

Contribution

The paper presents the first neural network for symmetry detection trained on extensive human-labeled data, capturing diverse symmetry types and outperforming prior methods.

Findings

Sym-NET outperforms existing algorithms on benchmark datasets.

It can identify 3D, occluded, and semantic symmetries.

High inter-person accuracy in human symmetry perception was leveraged.

Abstract

Humans take advantage of real world symmetries for various tasks, yet capturing their superb symmetry perception mechanism with a computational model remains elusive. Motivated by a new study demonstrating the extremely high inter-person accuracy of human perceived symmetries in the wild, we have constructed the first deep-learning neural network for reflection and rotation symmetry detection (Sym-NET), trained on photos from MS-COCO (Microsoft-Common Object in COntext) dataset with nearly 11K consistent symmetry-labels from more than 400 human observers. We employ novel methods to convert discrete human labels into symmetry heatmaps, capture symmetry densely in an image and quantitatively evaluate Sym-NET against multiple existing computer vision algorithms. On CVPR 2013 symmetry competition testsets and unseen MS-COCO photos, Sym-NET significantly outperforms all other competitors. Beyond mathematically well-defined symmetries on a plane, Sym-NET demonstrates abilities to identify viewpoint-varied 3D symmetries, partially occluded symmetrical objects, and symmetries at a semantic level.