Online test administration results in students selecting more responses to multiple-choice-multiple-response items

TL;DR

This study investigates how online test administration influences student response patterns on multiple-response items, revealing increased response selections online and emphasizing the need for consistent administration modes in assessments.

Contribution

The paper highlights the impact of online testing on response behavior for MCMR items and recommends mode-specific comparisons for accurate assessment analysis.

Findings

Students select more responses to MCMR items online.

Online administration leads to increased 'clickiness' in responses.

Mode of test administration affects student response patterns.

Abstract

Multiple-choice multiple-response (MCMR) items (i.e., multiple-choice questions for which more than one response may be selected) can be a valuable tool for assessment. Like traditional multiple-choice single-response questions, they are easy to score, but MCMR items can provide more information about student thinking by probing multiple reasoning facets in a single problem context. In this paper, we discuss differences in performance on MCMR items that result from differences in administration method (paper vs. online). In particular, we find a tendency for ``clickiness'' in online administration: students choose more responses to MCMR items when taking the electronic version of the assessment. Based on these findings, we recommend that online administration be compared only to other online administrations and not to paper administrations when comparing student results on MCMR items. These results also suggest that MCMR items provide a unique opportunity to probe differences in online and on-paper administration of low-stakes assessments.