Classroom Final Exam: An Instructor-Tested Reasoning Benchmark

TL;DR

CFE-Bench is a challenging, authentic multimodal reasoning benchmark from university exams across STEM fields, revealing that current large language models still struggle with multi-step reasoning and maintaining correct intermediate states.

Contribution

The paper introduces CFE-Bench, a new authentic, multimodal reasoning benchmark from university exams, and provides diagnostic insights into the reasoning limitations of frontier language models.

Findings

Frontier models achieve around 55-60% accuracy on CFE-Bench.

Models often generate more reasoning steps than instructors, risking error accumulation.

Models struggle to reliably derive and maintain correct intermediate reasoning states.

Abstract

We introduce CFE-Bench (Classroom Final Exam), a multimodal benchmark for evaluating the reasoning capabilities of large language models across more than 20 STEM domains. CFE-Bench is curated from repeatedly used, authentic university homework and exam problems, paired with reference solutions provided by course instructors. CFE-Bench remains challenging for frontier models: the newly released Gemini-3.1-pro-preview achieves 59.69% overall accuracy, while the second-best model, Gemini-3-flash-preview, reaches 55.46%, leaving substantial room for improvement. Beyond aggregate scores, we conduct a diagnostic analysis by decomposing instructor reference solutions into structured reasoning flows. We find that while frontier models often answer intermediate sub-questions correctly, they struggle to reliably derive and maintain correct intermediate states throughout multi-step solutions. We further observe that model-generated solutions typically contain more reasoning steps than instructor solutions, indicating lower step efficiency and a higher risk of error accumulation. Data and code are available at https://github.com/Analogy-AI/CFE_Bench.