Hybrid Instructor Ai Assessment In Academic Projects: Efficiency, Equity, And Methodological Lessons

TL;DR

This study demonstrates that a hybrid AI-instructor assessment system significantly improves grading efficiency, feedback quality, and fairness in large enrollment technical courses, while reducing faculty workload and maintaining high reliability.

Contribution

The paper introduces a supervised generative AI assessment system for academic reports, showcasing its effectiveness in efficiency, quality, and fairness improvements over traditional methods.

Findings

88% reduction in grading time

100% rubric coverage in feedback

No bias related to report length

Abstract

In technical subjects characterized by high enrollment, such as Basic Hydraulics, the assessment of reports necessitates superior levels of objectivity, consistency, and formative feedback; goals often compromised by faculty workload. This study presents the implementation of a generative artificial intelligence (AI) assisted assessment system, supervised by instructors, to grade 33 hydraulics reports. The central objective was to quantify its impact on the efficiency, quality, and fairness of the process. The employed methodology included the calibration of the Large Language Model (LLM) with a detailed rubric, the batch processing of assignments, and a human-in-the-loop validation phase. The quantitative results revealed a noteworthy 88% reduction in grading time (from 50 to 6 minutes per report, including verification) and a 733% increase in productivity. The quality of feedback was substantially improved, evidenced by 100% rubric coverage and a 150% increase in the anchoring of comments to textual evidence. The system proved to be equitable, exhibiting no bias related to report length, and highly reliable post-calibration (r = 0.96 between scores). It is concluded that the hybrid AI-instructor model optimizes the assessment process, thereby liberating time for high-value pedagogical tasks and enhancing the fairness and quality of feedback, in alignment with UNESCO's principles on the ethical use of AI in education.