Distilling ChatGPT for Explainable Automated Student Answer Assessment
Jiazheng Li, Lin Gui, Yuxiang Zhou, David West, Cesare Aloisi, Yulan, He

TL;DR
This paper presents a method to distill ChatGPT into a smaller model for explainable automated student answer assessment, improving scoring accuracy and rationale quality.
Contribution
The paper introduces a novel framework that fine-tunes a smaller language model using ChatGPT-generated rationales for better explainability and accuracy in student answer assessment.
Findings
11% improvement in QWK score over ChatGPT
Generated rationales are comparable to ChatGPT's
Effective approach for explainable automated assessment
Abstract
Providing explainable and faithful feedback is crucial for automated student answer assessment. In this paper, we introduce a novel framework that explores using ChatGPT, a cutting-edge large language model, for the concurrent tasks of student answer scoring and rationale generation. We identify the appropriate instructions by prompting ChatGPT with different templates to collect the rationales, where inconsistent rationales are refined to align with marking standards. The refined ChatGPT outputs enable us to fine-tune a smaller language model that simultaneously assesses student answers and provides rationales. Extensive experiments on the benchmark dataset show that the proposed method improves the overall QWK score by 11% compared to ChatGPT. Furthermore, our thorough analysis and human evaluation demonstrate that the rationales generated by our proposed method are comparable to…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Online Learning and Analytics · Multimodal Machine Learning Applications
MethodsALIGN
