PhysicsArena: The First Multimodal Physics Reasoning Benchmark Exploring Variable, Process, and Solution Dimensions

TL;DR

PhysicsArena is a novel multimodal benchmark designed to evaluate large language models' physics reasoning across variable identification, process formulation, and solution derivation, addressing limitations of existing text-only physics assessments.

Contribution

It introduces the first comprehensive multimodal physics reasoning benchmark covering multiple reasoning dimensions, filling a gap in current evaluation tools.

Findings

Establishes a new standard for multimodal physics reasoning evaluation.

Highlights the challenges faced by current MLLMs in physics tasks.

Provides a platform for future development of more capable physics reasoning models.

Abstract

Multimodal Large Language Models (MLLMs) have demonstrated remarkable capabilities in diverse reasoning tasks, yet their application to complex physics reasoning remains underexplored. Physics reasoning presents unique challenges, requiring grounding in physical conditions and the interpretation of multimodal information. Current physics benchmarks are limited, often focusing on text-only inputs or solely on problem-solving, thereby overlooking the critical intermediate steps of variable identification and process formulation. To address these limitations, we introduce PhysicsArena, the first multimodal physics reasoning benchmark designed to holistically evaluate MLLMs across three critical dimensions: variable identification, physical process formulation, and solution derivation. PhysicsArena aims to provide a comprehensive platform for assessing and advancing the multimodal physics reasoning abilities of MLLMs.