Evaluating Micro Parsons Problems as Exam Questions

TL;DR

This study evaluates the effectiveness and student perceptions of micro Parsons problems as exam questions in an introductory Python programming course, using Item Response Theory and survey analysis.

Contribution

It provides empirical evidence on the difficulty, discriminative ability, and student perceptions of micro Parsons problems as exam items, filling a research gap.

Findings

Micro Parsons problems have comparable difficulty to traditional coding questions.

They effectively differentiate between high and low ability students.

Students perceive micro Parsons as helpful for exam preparation.

Abstract

Parsons problems are a type of programming activity that present learners with blocks of existing code and requiring them to arrange those blocks to form a program rather than write the code from scratch. Micro Parsons problems extend this concept by having students assemble segments of code to form a single line of code rather than an entire program. Recent investigations into micro Parsons problems have primarily focused on supporting learners leaving open the question of micro Parsons efficacy as an exam item and how students perceive it when preparing for exams. To fill this gap, we included a variety of micro Parsons problems on four exams in an introductory programming course taught in Python. We use Item Response Theory to investigate the difficulty of the micro Parsons problems as well as the ability of the questions to differentiate between high and low ability students. We then compare these results to results for related questions where students are asked to write a single line of code from scratch. Finally, we conduct a thematic analysis of the survey responses to investigate how students' perceptions of micro Parsons both when practicing for exams and as they appear on exams.