Pensieve Grader: An AI-Powered, Ready-to-Use Platform for Effortless Handwritten STEM Grading

TL;DR

Pensieve is an AI-powered platform that streamlines handwritten STEM response grading by integrating large language models into a human-in-the-loop system, significantly reducing grading time while maintaining high accuracy.

Contribution

This paper introduces Pensieve, a comprehensive AI-assisted grading platform supporting the entire grading pipeline with real-world deployment and empirical validation across multiple STEM disciplines.

Findings

Reduces grading time by 65% on average

Achieves 95.4% agreement with instructor grades

Supports over 300,000 student responses across 20 institutions

Abstract

Grading handwritten, open-ended responses remains a major bottleneck in large university STEM courses. We introduce Pensieve (https://www.pensieve.co), an AI-assisted grading platform that leverages large language models (LLMs) to transcribe and evaluate student work, providing instructors with rubric-aligned scores, transcriptions, and confidence ratings. Unlike prior tools that focus narrowly on specific tasks like transcription or rubric generation, Pensieve supports the entire grading pipeline-from scanned student submissions to final feedback-within a human-in-the-loop interface. Pensieve has been deployed in real-world courses at over 20 institutions and has graded more than 300,000 student responses. We present system details and empirical results across four core STEM disciplines: Computer Science, Mathematics, Physics, and Chemistry. Our findings show that Pensieve reduces grading time by an average of 65%, while maintaining a 95.4% agreement rate with instructor-assigned grades for high-confidence predictions.