From Flat to Structural: Enhancing Automated Short Answer Grading with GraphRAG

TL;DR

This paper introduces GraphRAG, a structured knowledge graph approach for automated short answer grading that improves reasoning and accuracy over traditional flat retrieval methods.

Contribution

The paper presents a novel GraphRAG framework that models knowledge as a graph and employs neurosymbolic algorithms for better reasoning in educational content assessment.

Findings

GraphRAG significantly outperforms standard RAG baselines.

Structural retrieval improves reasoning in science education assessment.

HippoRAG enhances evaluation of Science and Engineering Practices.

Abstract

Automated short answer grading (ASAG) is critical for scaling educational assessment, yet large language models (LLMs) often struggle with hallucinations and strict rubric adherence due to their reliance on generalized pre-training. While Rretrieval-Augmented Generation (RAG) mitigates these issues, standard "flat" vector retrieval mechanisms treat knowledge as isolated fragments, failing to capture the structural relationships and multi-hop reasoning essential for complex educational content. To address this limitation, we introduce a Graph Retrieval-Augmented Generation (GraphRAG) framework that organizes reference materials into a structured knowledge graph to explicitly model dependencies between concepts. Our methodology employs a dual-phase pipeline: utilizing Microsoft GraphRAG for high-fidelity graph construction and the HippoRAG neurosymbolic algorithm to execute associative graph traversals, thereby retrieving comprehensive, connected subgraphs of evidence. Experimental evaluations on a Next Generation Science Standards (NGSS) dataset demonstrate that this structural approach significantly outperforms standard RAG baselines across all metrics. Notably, the HippoRAG implementation achieved substantial improvements in evaluating Science and Engineering Practices (SEP), confirming the superiority of structural retrieval in verifying the logical reasoning chains required for higher-order academic assessment.