ViSRA: A Video-based Spatial Reasoning Agent for Multi-modal Large Language Models

TL;DR

ViSRA introduces a training-free, modular framework for probing and enhancing 3D spatial reasoning in multi-modal large language models using explicit spatial information.

Contribution

The paper presents ViSRA, a plug-and-play, human-aligned spatial reasoning agent that improves MLLMs without additional training or curated datasets.

Findings

ViSRA improves MLLMs performance on spatial reasoning benchmarks by up to 15.6%.

ViSRA enhances generalization to unseen 3D spatial tasks with up to 28.9% improvement.

The approach eliminates the need for post-training and manual spatial dataset curation.

Abstract

Recent advances in Multi-modal Large Language Models (MLLMs) target 3D spatial intelligence, yet the progress has been largely driven by post-training on curated benchmarks, leaving the inference-time approach relatively underexplored. In this paper, we take a training-free perspective and introduce ViSRA, a human-aligned Video-based Spatial Reasoning Agent, as a framework to probe the spatial reasoning mechanism of MLLMs. ViSRA elicits spatial reasoning in a modular and extensible manner by leveraging explicit spatial information from expert models, enabling a plug-and-play flexible paradigm. ViSRA offers two key advantages: (1) human-aligned and transferable 3D understanding rather than task-specific overfitting; and (2) no post-training computational cost along with heavy manual curation of spatial reasoning datasets. Experimental results demonstrate consistent improvement across a set of MLLMs on both existing benchmarks and unseen 3D spatial reasoning tasks, with ViSRA outperforming baselines by up to a 15.6% and 28.9% absolute margin respectively.