Unraveling the geometry of visual relational reasoning

TL;DR

This paper introduces SimplifiedRPM, a new benchmark for evaluating abstract relational reasoning in neural networks, and analyzes how different models, especially SCL, generalize and align with human reasoning through geometric and layer-wise analysis.

Contribution

It presents SimplifiedRPM as a novel benchmark, compares multiple models including SCL, and develops a geometric framework to understand and improve relational reasoning in AI.

Findings

SCL best generalizes and aligns with human behavior

A geometric trade-off between signal and dimensionality affects generalization

Layer-wise analysis reveals where relational structure emerges

Abstract

Humans readily generalize abstract relations, such as recognizing "constant" in shape or color, whereas neural networks struggle, limiting their flexible reasoning. To investigate mechanisms underlying such generalization, we introduce SimplifiedRPM, a novel benchmark for systematically evaluating abstract relational reasoning, addressing limitations in prior datasets. In parallel, we conduct human experiments to quantify relational difficulty, enabling direct model-human comparisons. Testing four models, ResNet-50, Vision Transformer, Wild Relation Network, and Scattering Compositional Learner (SCL), we find that SCL generalizes best and most closely aligns with human behavior. Using a geometric approach, we identify key representation properties that accurately predict generalization and uncover a fundamental trade-off between signal and dimensionality: novel relations compress into training-induced subspaces. Layer-wise analysis reveals where relational structure emerges, highlights bottlenecks, and generates concrete hypotheses about abstract reasoning in the brain. Motivated by these insights, we propose SNRloss, a novel objective explicitly balancing representation geometry. Our results establish a geometric foundation for relational reasoning, paving the way for more human-like visual reasoning in AI and opening promising avenues for extending geometric analysis to broader cognitive tasks.