Trading off performance and human oversight in algorithmic policy: evidence from Danish college admissions

TL;DR

This study evaluates AI models for predicting student success in Danish college admissions, highlighting improved accuracy and fairness but also noting reduced transparency and oversight, with implications for policy and efficiency.

Contribution

It demonstrates that advanced sequential AI models outperform traditional methods in predicting college completion, offering insights into balancing performance with human oversight.

Findings

AI models improve prediction accuracy over traditional methods

Advanced models better match students to programs, enhancing efficiency

Using AI in admissions can yield economic benefits despite reduced oversight

Abstract

Student dropout is a significant concern for educational institutions due to its social and economic impact, driving the need for risk prediction systems to identify at-risk students before enrollment. We explore the accuracy of such systems in the context of higher education by predicting degree completion before admission, with potential applications for prioritizing admissions decisions. Using a large-scale dataset from Danish higher education admissions, we demonstrate that advanced sequential AI models offer more precise and fair predictions compared to current practices that rely on either high school grade point averages or human judgment. These models not only improve accuracy but also outperform simpler models, even when the simpler models use protected sociodemographic attributes. Importantly, our predictions reveal how certain student profiles are better matched with specific programs and fields, suggesting potential efficiency and welfare gains in public policy. We estimate that even the use of simple AI models to guide admissions decisions, particularly in response to a newly implemented nationwide policy reducing admissions by 10 percent, could yield significant economic benefits. However, this improvement would come at the cost of reduced human oversight and lower transparency. Our findings underscore both the potential and challenges of incorporating advanced AI into educational policymaking.