A review and evaluation of secondary school accountability in England: Statistical strengths, weaknesses, and challenges for 'Progress 8'

TL;DR

This paper critically reviews the Progress 8 secondary school accountability system in England, analyzing its statistical strengths and weaknesses, and offers recommendations to improve its effectiveness amid challenges like COVID-19.

Contribution

It provides a comprehensive evaluation of Progress 8's statistical methods, limitations, and challenges, proposing practical improvements for school accountability.

Findings

Progress 8 has issues with statistical uncertainty and school performance volatility.

Adjustments for pupil background characteristics are limited.

COVID-19 posed significant challenges to the system's reliability.

Abstract

School performance measures are published annually in England to hold schools to account and to support parental school choice. This article reviews and evaluates the Progress 8 secondary school accountability system for state-funded schools. We assess the statistical strengths and weaknesses of Progress 8 relating to: choice of pupil outcome attainment measures; potential adjustments for pupil input attainment and background characteristics; decisions around which schools and pupils are excluded from the measure; presentation of Progress 8 to users, choice of statistical model, and calculation of statistical uncertainty; and issues related to the volatility of school performance over time, including scope for reporting multi-year averages. We then discuss challenges for Progress 8 raised by the COVID-19 pandemic. Six simple recommendations follow to improve Progress 8 and school accountability in England.