SPARC: Separating Perception And Reasoning Circuits for Test-time Scaling of VLMs

TL;DR

SPARC introduces a modular approach that separates perception and reasoning in vision-language models, enabling more flexible, efficient, and accurate test-time scaling and adaptation for visual reasoning tasks.

Contribution

It proposes a novel framework that decouples perception from reasoning, allowing independent scaling, optimization, and efficient processing in vision-language models.

Findings

Outperforms monolithic baselines on visual reasoning benchmarks.

Improves Qwen3VL-4B accuracy by 6.7 percentage points on V* VQA.

Reduces token budget by 200x while maintaining or improving performance.

Abstract

Despite recent successes, test-time scaling - i.e., dynamically expanding the token budget during inference as needed - remains brittle for vision-language models (VLMs): unstructured chains-of-thought about images entangle perception and reasoning, leading to long, disorganized contexts where small perceptual mistakes may cascade into completely wrong answers. Moreover, expensive reinforcement learning with hand-crafted rewards is required to achieve good performance. Here, we introduce SPARC (Separating Perception And Reasoning Circuits), a modular framework that explicitly decouples visual perception from reasoning. Inspired by sequential sensory-to-cognitive processing in the brain, SPARC implements a two-stage pipeline where the model first performs explicit visual search to localize question-relevant regions, then conditions its reasoning on those regions to produce the final answer. This separation enables independent test-time scaling with asymmetric compute allocation (e.g., prioritizing perceptual processing under distribution shift), supports selective optimization (e.g., improving the perceptual stage alone when it is the bottleneck for end-to-end performance), and accommodates compressed contexts by running global search at lower image resolutions and allocating high-resolution processing only to selected regions, thereby reducing total visual tokens count and compute. Across challenging visual reasoning benchmarks, SPARC outperforms monolithic baselines and strong visual-grounding approaches. For instance, SPARC improves the accuracy of Qwen3VL-4B on the $V^*$ VQA benchmark by 6.7 percentage points, and it surpasses "thinking with images" by 4.6 points on a challenging OOD task despite requiring a 200$\times$ lower token budget.