EduBench: A Comprehensive Benchmarking Dataset for Evaluating Large Language Models in Diverse Educational Scenarios

TL;DR

EduBench introduces a diverse, educational-specific benchmark dataset with multi-dimensional evaluation metrics, enabling better assessment and development of language models in educational contexts, with promising results from training smaller models.

Contribution

This paper presents the first comprehensive educational benchmark dataset and evaluation framework, along with a small-scale model achieving competitive performance to larger models.

Findings

The dataset covers 9 major educational scenarios and over 4,000 contexts.

Proposed metrics evaluate 12 critical aspects for educational language models.

Small models trained on EduBench perform comparably to state-of-the-art large models.

Abstract

As large language models continue to advance, their application in educational contexts remains underexplored and under-optimized. In this paper, we address this gap by introducing the first diverse benchmark tailored for educational scenarios, incorporating synthetic data containing 9 major scenarios and over 4,000 distinct educational contexts. To enable comprehensive assessment, we propose a set of multi-dimensional evaluation metrics that cover 12 critical aspects relevant to both teachers and students. We further apply human annotation to ensure the effectiveness of the model-generated evaluation responses. Additionally, we succeed to train a relatively small-scale model on our constructed dataset and demonstrate that it can achieve performance comparable to state-of-the-art large models (e.g., Deepseek V3, Qwen Max) on the test set. Overall, this work provides a practical foundation for the development and evaluation of education-oriented language models. Code and data are released at https://github.com/ybai-nlp/EduBench.