Personalized Stopping Rules in Bayesian Adaptive Mastery Assessment

TL;DR

This paper introduces Bayesian Adaptive Mastery Assessment (BAMA), a novel model that efficiently estimates student mastery by integrating response accuracy and time, optimizing assessment length and certainty through a Bayesian decision framework.

Contribution

The paper presents BAMA, a new adaptive assessment model that dynamically infers mastery levels using response data and Bayesian decision theory, improving efficiency and accuracy.

Findings

BAMA converges with low variance and high efficiency.

Assessment duration is shortened for all students.

The model effectively balances assessment length and mastery certainty.

Abstract

We propose a new model to assess the mastery level of a given skill efficiently. The model, called Bayesian Adaptive Mastery Assessment (BAMA), uses information on the accuracy and the response time of the answers given and infers the mastery at every step of the assessment. BAMA balances the length of the assessment and the certainty of the mastery inference by employing a Bayesian decision-theoretic framework adapted to each student. All these properties contribute to a novel approach in assessment models for intelligent learning systems. The purpose of this research is to explore the properties of BAMA and evaluate its performance concerning the number of questions administered and the accuracy of the final mastery estimates across different students. We simulate student performances and establish that the model converges with low variance and high efficiency leading to shorter assessment duration for all students. Considering the experimental results, we expect our approach to avoid the issue of over-practicing and under-practicing and facilitate the development of Learning Analytics tools to support the tutors in the evaluation of learning effects and instructional decision making.