Representing uncertainty on model analysis plots

TL;DR

This paper introduces a method to incorporate uncertainty into model analysis plots for student learning, enhancing the visualization of measurement reliability in educational assessments.

Contribution

It extends existing model plots by adding error bars to represent measurement uncertainty, providing a clearer understanding of data reliability.

Findings

Error bars effectively visualize uncertainty in model analysis.

The method improves interpretation of student learning data.

A template facilitates easy implementation of the enhanced plots.

Abstract

Model analysis provides a mechanism for representing student learning as measured by standard multiple-choice surveys. The model plot contains information regarding both how likely students in a particular class are to choose the correct answer and how likely they are to choose an answer consistent with a well-documented conceptual model. Unfortunately Bao's original presentation of the model plot did not include a way to represent uncertainty in these measurements. I present details of a method to add error bars to model plots by expanding the work of Sommer and Lindell. I also provide a template for generating model plots with error bars.