Assessing the quality of a student-generated question repository

TL;DR

This study evaluates the quality of student-generated questions and explanations in physics courses, finding that most questions surpass basic recall and explanations are generally high quality, influenced by scaffolding activities.

Contribution

It provides empirical evidence that students can produce high-quality questions and explanations in physics, especially with scaffolding support, contrasting with prior studies in other subjects.

Findings

Over 75% of questions met quality criteria

Approximately 60% of explanations were high or outstanding

Most questions were beyond simple recall

Abstract

We present results from a study that categorizes and assesses the quality of questions and explanations authored by students, in question repositories produced as part of the summative assessment in introductory physics courses over the past two years. Mapping question quality onto the levels in the cognitive domain of Bloom's taxonomy, we find that students produce questions of high quality. More than three-quarters of questions fall into categories beyond simple recall, in contrast to similar studies of student-authored content in different subject domains. Similarly, the quality of student-authored explanations for questions was also high, with approximately 60% of all explanations classified as being of high or outstanding quality. Overall, 75% of questions met combined quality criteria, which we hypothesize is due in part to the in-class scaffolding activities that we provided for students ahead of requiring them to author questions.