Online administration of a reasoning inventory in development

TL;DR

This paper discusses the rapid transition to online, unproctored administration of a physics reasoning assessment during the COVID-19 pandemic, comparing its results to traditional in-person testing.

Contribution

It provides insights into best practices and preliminary findings on online administration of a physics reasoning inventory in a large-enrollment course.

Findings

Little difference between online and in-person results

Online administration can effectively assess reasoning

Use of multiple-response items probes reasoning facets

Abstract

We are developing a new research based assessment (RBA) focused on quantitative reasoning -- rather than conceptual understanding -- in physics contexts. We rapidly moved administration of the RBA online in Spring 2020 due to the COVID-19 pandemic. We present our experiences with online, unproctored administration of an RBA in development to students enrolled in a large-enrollment, calculus-based, introductory physics course. We describe our attempts to adhere to best practices on a limited time frame, and present a preliminary analysis of the results, comparing results from the online administration to earlier results from in-person, proctored administration. We include discussion of online administration of multiple-choice/multiple-response (MCMR) items, which we use on the instrument as a way to probe multiple facets of student reasoning. Our initial comparison indicates little difference between online and paper administrations of the RBA, consistent with previous work by other researchers.