A Comparison of Value-Added Models for School Accountability

TL;DR

This paper compares different value-added models used in school accountability, analyzing their effectiveness and differences through a case study of England's 'Progress 8' system, highlighting implications for transparency and fairness.

Contribution

It provides an empirical comparison of simple and complex value-added models in school accountability, focusing on their similarities and differences in a real-world case.

Findings

Simple models often produce similar school effects to complex models

Adjustments for sociodemographics have limited impact on rankings

Fewer adjustments may be justified for transparency and data limitations

Abstract

School accountability systems increasingly hold schools to account for their performances using value-added models purporting to measure the effects of schools on student learning. The most common approach is to fit a linear regression of student current achievement on student prior achievement, where the school effects are the school means of the predicted residuals. In the literature, further adjustments are usually made for student sociodemographics and sometimes school composition and 'non-malleable' characteristics. However, accountability systems typically make fewer adjustments: for transparency to end users, because data is unavailable or of insufficient quality, or for ideological reasons. There is therefore considerable interest in understanding the extent to which simpler models give similar school effects to more theoretically justified but complex models. We explore these issues via a case study and empirical analysis of England's 'Progress 8' secondary school accountability system.