From Local Cues to Global Percepts: Emergent Gestalt Organization in Self-Supervised Vision Models

TL;DR

This paper demonstrates that self-supervised vision models, especially Vision Transformers trained with Masked Autoencoding, develop Gestalt-like global perceptual organization, and introduces a diagnostic test to evaluate this ability.

Contribution

The study reveals how specific training methods induce Gestalt principles in vision models and introduces DiSRT to assess global structure sensitivity.

Findings

ViTs with MAE show Gestalt law-like activation patterns.

Self-supervised models outperform supervised ones on DiSRT.

Sparsity mechanisms can restore global sensitivity in models.

Abstract

Human vision organizes local cues into coherent global forms using Gestalt principles like closure, proximity, and figure-ground assignment -- functions reliant on global spatial structure. We investigate whether modern vision models show similar behaviors, and under what training conditions these emerge. We find that Vision Transformers (ViTs) trained with Masked Autoencoding (MAE) exhibit activation patterns consistent with Gestalt laws, including illusory contour completion, convexity preference, and dynamic figure-ground segregation. To probe the computational basis, we hypothesize that modeling global dependencies is necessary for Gestalt-like organization. We introduce the Distorted Spatial Relationship Testbench (DiSRT), which evaluates sensitivity to global spatial perturbations while preserving local textures. Using DiSRT, we show that self-supervised models (e.g., MAE, CLIP) outperform supervised baselines and sometimes even exceed human performance. ConvNeXt models trained with MAE also exhibit Gestalt-compatible representations, suggesting such sensitivity can arise without attention architectures. However, classification finetuning degrades this ability. Inspired by biological vision, we show that a Top-K activation sparsity mechanism can restore global sensitivity. Our findings identify training conditions that promote or suppress Gestalt-like perception and establish DiSRT as a diagnostic for global structure sensitivity across models.