Developing Consistency Among Undergraduate Graders Scoring Open-Ended Statistics Tasks

TL;DR

This study shows that undergraduate graders can achieve consistent scoring of open-ended statistics tasks through proper training and high-quality rubrics, improving grading reliability in higher education.

Contribution

It provides evidence that undergraduate students can be trained to reliably score open-ended questions, highlighting the importance of training and rubric quality.

Findings

Undergraduate graders can develop substantial agreement with instructors.

Proper training enhances scoring consistency.

High-quality rubrics are crucial for reliable grading.

Abstract

Undergraduate graders are frequently important contributors to the teaching team in post-secondary education settings. This study set out to investigate agreement for a team of undergraduate graders as they acquired training and experience for scoring responses to open-ended tasks. Results demonstrate compelling evidence that undergraduate students can develop the ability to establish and sustain substantial agreement with an instructor, especially when equipped with proper training and a high-quality scoring rubric.