# 11Plus-Bench: Demystifying Multimodal LLM Spatial Reasoning with Cognitive-Inspired Analysis

## TL;DR

This paper introduces 11Plus-Bench, a benchmark for evaluating multimodal large language models' spatial reasoning, revealing emerging abilities and limitations compared to human spatial cognition through detailed analysis.

## Contribution

The paper presents a new benchmark and evaluation framework for assessing spatial reasoning in multimodal LLMs, with fine-grained annotations and extensive experiments.

## Key findings

- MLLMs show early signs of spatial cognition.
- Performance correlates with reasoning complexity but remains largely random.
- Humans outperform models with predictable correctness based on pattern complexity.

## Abstract

For human cognitive process, spatial reasoning and perception are closely entangled, yet the nature of this interplay remains underexplored in the evaluation of multimodal large language models (MLLMs). While recent MLLM advancements show impressive performance on reasoning, their capacity for human-like spatial cognition remains an open question. In this work, we introduce a systematic evaluation framework to assess the spatial reasoning abilities of state-of-the-art MLLMs relative to human performance. Central to our work is 11Plus-Bench, a high-quality benchmark derived from realistic standardized spatial aptitude tests. 11Plus-Bench also features fine-grained expert annotations of both perceptual complexity and reasoning process, enabling detailed instance-level analysis of model behavior. Through extensive experiments across 14 MLLMs and human evaluation, we find that current MLLMs exhibit early signs of spatial cognition. Despite a large performance gap compared to humans, MLLMs' cognitive profiles resemble those of humans in that cognitive effort correlates strongly with reasoning-related complexity. However, instance-level performance in MLLMs remains largely random, whereas human correctness is highly predictable and shaped by abstract pattern complexity. These findings highlight both emerging capabilities and limitations in current MLLMs' spatial reasoning capabilities and provide actionable insights for advancing model design.