MMCL-Bench: Multimodal Context Learning from Visual Rules, Procedures, and Evidence

TL;DR

MMCL-Bench is a new benchmark designed to evaluate multimodal context learning, challenging models to recover and reason over visual evidence across diverse tasks, revealing significant gaps in current system capabilities.

Contribution

This paper introduces MMCL-Bench, a comprehensive benchmark for multimodal context learning from visual data, highlighting current models' limitations and guiding future research.

Findings

Current models solve fewer than one-third of tasks under strict evaluation.

Failures occur across context anchoring, evidence extraction, reasoning, and response construction.

MMCL-Bench exposes critical bottlenecks in multimodal context learning capabilities.

Abstract

We introduce MMCL-Bench, a benchmark for multimodal context learning: learning task-local rules, procedures, and empirical patterns from visual or mixed-modality teaching context and applying them to new visual instances. Unlike text-only context learning or standard multimodal question answering, this setting requires models to recover and localize relevant evidence from images, screenshots, manuals, videos, and frame sequences before they can reason over the learned context. MMCL-Bench contains 102 tasks spanning three categories: rule system application, procedural task execution, and empirical discovery and induction. We evaluate frontier multimodal models with strict rubric-based scoring and find that current systems remain far from robust multimodal context learning, with even the strongest model solving fewer than one-third of tasks under strict evaluation. Diagnostic ablations and error analysis show that failures arise throughout the context-to-answer pipeline, including context anchoring, visual evidence extraction, context reasoning, and response construction. MMCL-Bench thus highlights multimodal context learning as an important unsolved capability bottleneck for current multimodal models.