Improving Graduation Rate Estimates Using Regularly Updating Multi-Level Absorbing Markov Chains

TL;DR

This paper introduces a regularly updating multi-level absorbing Markov chain method to improve the accuracy and reduce bias in university graduation rate estimates, especially for small cohorts.

Contribution

The paper proposes and empirically evaluates a novel RUML-AMC approach that updates transition matrices annually to enhance estimation accuracy and reduce bias in graduation rate calculations.

Findings

RUML-AMC nearly eliminates estimation bias.

Reduces estimation variation by over 40%.

Improves accuracy for small sample sizes.

Abstract

American universities use a procedure based on a rolling six-year graduation rate to calculate statistics regarding their students' final educational outcomes (graduating or not graduating). As~an alternative to the six-year graduation rate method, many studies have applied absorbing Markov chains for estimating graduation rates. In both cases, a frequentist approach is used. For~the standard six-year graduation rate method, the frequentist approach corresponds to counting the number of students who finished their program within six years and dividing by the number of students who entered that year. In the case of absorbing Markov chains, the frequentist approach is used to compute the underlying transition matrix, which is then used to estimate the graduation rate. In this paper, we apply a sensitivity analysis to compare the performance of the standard six-year graduation rate method with that of absorbing Markov chains. Through the analysis, we highlight significant limitations with regards to the estimation accuracy of both approaches when applied to small sample sizes or cohorts at a university. Additionally, we note that the Absorbing Markov chain method introduces a significant bias, which leads to an underestimation of the true graduation rate. To~overcome both these challenges, we propose and evaluate the use of a regularly updating multi-level absorbing Markov chain (RUML-AMC) in which the transition matrix is updated year to year. We empirically demonstrate that the proposed RUML-AMC approach nearly eliminates estimation bias while reducing the estimation variation by more than 40%, especially for populations with small sample sizes.