Automatic Scoring of Students' Science Writing Using Hybrid Neural Network

TL;DR

This paper presents a hybrid neural network model that effectively scores students' science responses with accuracy comparable to BERT but with greater efficiency, outperforming traditional machine learning methods.

Contribution

The study introduces a multi-perspective hybrid neural network (HNN) for automatic scoring, demonstrating improved accuracy and efficiency over existing ML models in science education assessment.

Findings

HNN achieved higher accuracy than Naive Bayes, Logistic Regression, AACR, and BERT.

HNN's accuracy (96.23%) is comparable to BERT's (96.12%).

HNN is twice as efficient as BERT in training and inference.

Abstract

This study explores the efficacy of a multi-perspective hybrid neural network (HNN) for scoring student responses in science education with an analytic rubric. We compared the accuracy of the HNN model with four ML approaches (BERT, AACR, Naive Bayes, and Logistic Regression). The results have shown that HHN achieved 8%, 3%, 1%, and 0.12% higher accuracy than Naive Bayes, Logistic Regression, AACR, and BERT, respectively, for five scoring aspects (p<0.001). The overall HNN's perceived accuracy (M = 96.23%, SD = 1.45%) is comparable to the (training and inference) expensive BERT model's accuracy (M = 96.12%, SD = 1.52%). We also have observed that HNN is x2 more efficient in training and inferencing than BERT and has comparable efficiency to the lightweight but less accurate Naive Bayes model. Our study confirmed the accuracy and efficiency of using HNN to score students' science writing automatically.