Correcting Sample Selection Bias in PISA Rankings

TL;DR

This paper develops a method to correct for sample selection bias in PISA rankings, revealing that accounting for dropout-related survival bias significantly alters country performance comparisons.

Contribution

It introduces a tailored Heckman selection model adjustment for fully truncated data, improving the accuracy of international educational performance assessments.

Findings

Adjusting for bias changes country rankings substantially.

The method leverages joint normality of errors and selection rates.

Results emphasize importance of correcting for dropout bias.

Abstract

This paper addresses the critical issue of sample selection bias in cross-country comparisons based on international assessments such as the Programme for International Student Assessment (PISA). Although PISA is widely used to benchmark educational performance across countries, it samples only students who remain enrolled in school at age 15. This introduces survival bias, particularly in countries with high dropout rates, potentially leading to distorted comparisons. To correct for this bias, I develop a simple adjustment of the classical Heckman selection model tailored to settings with fully truncated outcome data. My approach exploits the joint normality of latent errors and leverages information on the selection rate, allowing identification of the counterfactual mean outcome for the full population of 15-year-olds. Applying this method to PISA 2018 data, I show that adjusting for selection bias results in substantial changes in country rankings based on average performance. These results highlight the importance of accounting for non-random sample selection to ensure accurate and policy-relevant international comparisons of educational outcomes.