Partial Identification of Answer Reviewing Effects in Multiple-Choice Exams

TL;DR

This paper investigates whether reviewing answers during multiple-choice exams improves scores, using a causal framework to address identification challenges and providing partial insights into the effect, supported by analysis of real assessment data.

Contribution

It introduces partial identification methods to infer the impact of answer reviewing in exams, overcoming data and treatment ambiguity challenges.

Findings

Reviewing answers generally benefits examinees' scores.

Partial identification can infer the sign of the reviewing effect.

Empirical analysis supports reviewing as beneficial.

Abstract

Does reviewing previous answers during multiple-choice exams help examinees increase their final score? This article formalizes the question using a rigorous causal framework, the potential outcomes framework. Viewing examinees' reviewing status as a treatment and their final score as an outcome, the article first explains the challenges of identifying the causal effect of answer reviewing in regular exam-taking settings. In addition to the incapability of randomizing the treatment selection (reviewing status) and the lack of other information to make this selection process ignorable, the treatment variable itself is not fully known to researchers. Looking at examinees' answer sheet data, it is unclear whether an examinee who did not change his or her answer on a specific item reviewed it but retained the initial answer (treatment condition) or chose not to review it (control condition). Despite such challenges, however, the article develops partial identification strategies and shows that the sign of the answer reviewing effect can be reasonably inferred. By analyzing a statewide math assessment data set, the article finds that reviewing initial answers is generally beneficial for examinees.