The Potential of Answer Classes in Large-scale Written Computer-Science Exams -- Vol. 2

TL;DR

This paper explores the use of answer classes in large-scale written computer science exams to analyze student responses, including misconceptions, improving assessment and teaching strategies, with a digital correction system demonstrated.

Contribution

It introduces a novel approach of developing answer classes for exam responses, demonstrating its effectiveness and potential for digital implementation in large-scale assessments.

Findings

Answer classes help identify misconceptions in student answers.

Instructors found the approach valuable despite initial reservations.

The digital system facilitates efficient correction based on answer classes.

Abstract

Students' answers to tasks provide a valuable source of information in teaching as they result from applying cognitive processes to a learning content addressed in the task. Due to steadily increasing course sizes, analyzing student answers is frequently the only means of obtaining evidence about student performance. However, in many cases, resources are limited, and when evaluating exams, the focus is solely on identifying correct or incorrect answers. This overlooks the value of analyzing incorrect answers, which can help improve teaching strategies or identify misconceptions to be addressed in the next cohort. In teacher training for secondary education, assessment guidelines are mandatory for every exam, including anticipated errors and misconceptions. We applied this concept to a university exam with 462 students and 41 tasks. For each task, the instructors developed answer classes -- classes of expected responses, to which student answers were mapped during the exam correction process. The experiment resulted in a shift in mindset among the tutors and instructors responsible for the course: after initially having great reservations about whether the significant additional effort would yield an appropriate benefit, the procedure was subsequently found to be extremely valuable. The concept presented, and the experience gained from the experiment were cast into a system with which it is possible to correct paper-based exams on the basis of answer classes. This updated version of the paper provides an overview and new potential in the course of using the digital version of the approach.