MATRIX: A Multimodal Benchmark and Post-Training Framework for Materials Science

TL;DR

MATRIX introduces a multimodal benchmark and post-training framework for materials science that leverages experimental images alongside text to improve scientific reasoning and interpretation of experimental data.

Contribution

The paper presents a new benchmark and demonstrates that incorporating visual data during post-training enhances reasoning accuracy in materials science tasks.

Findings

Visual supervision improves experimental interpretation by 10-25%.

Post-training with multimodal data yields 5-16% gains on text-only reasoning.

Cross-modal alignment is crucial for effective transfer.

Abstract

Scientific reasoning in materials science requires integrating multimodal experimental evidence with underlying physical theory. Existing benchmarks make it difficult to assess whether incorporating visual experimental data during post-training improves mechanism-grounded explanation reasoning beyond text-only supervision. We introduce MATRIX, a multimodal benchmark for materials science reasoning that evaluates foundational theory, research-level reasoning, and the interpretation of real experimental artifacts across multiple characterization modalities. Using MATRIX as a controlled diagnostic, we isolate the effect of visual grounding by comparing post-training on structured materials science text alone with post-training that incorporates paired experimental images. Despite using relatively small amounts of multimodal data, visual supervision improves experimental interpretation by 10-25% and yields 5-16% gains on text-only scientific reasoning tasks. Our results demonstrate that these improvements rely on correct image-text alignment during post-training, highlighting cross-modal representational transfer. We also observe consistent improvements on ScienceQA and PubMedQA, demonstrating that the benefits of structured multimodal post-training extend beyond materials science. The MATRIX dataset is available at https://huggingface.co/datasets/radical-ai/MATRIX and the model at https://huggingface.co/radical-ai/MATRIX-PT.