Explainable AI for Predicting and Understanding Mathematics Achievement: A Cross-National Analysis of PISA 2018

TL;DR

This study employs explainable AI models to predict and analyze mathematics achievement across ten countries using PISA 2018 data, revealing key socio-economic and attitudinal predictors and emphasizing model interpretability.

Contribution

It introduces the application of XAI techniques to cross-national educational data, comparing multiple models and identifying key predictors with interpretability tools.

Findings

Non-linear models outperform linear regression in accuracy.

Socio-economic status and attitudes are key predictors.

Model interpretability reveals country-specific factors.

Abstract

Understanding the factors that shape students' mathematics performance is vital for designing effective educational policies. This study applies explainable artificial intelligence (XAI) techniques to PISA 2018 data to predict math achievement and identify key predictors across ten countries (67,329 students). We tested four models: Multiple Linear Regression (MLR), Random Forest (RF), CATBoost, and Artificial Neural Networks (ANN), using student, family, and school variables. Models were trained on 70% of the data (with 5-fold cross-validation) and tested on 30%, stratified by country. Performance was assessed with R^2 and Mean Absolute Error (MAE). To ensure interpretability, we used feature importance, SHAP values, and decision tree visualizations. Non-linear models, especially RF and ANN, outperformed MLR, with RF balancing accuracy and generalizability. Key predictors included socio-economic status, study time, teacher motivation, and students' attitudes toward mathematics, though their impact varied across countries. Visual diagnostics such as scatterplots of predicted vs actual scores showed RF and CATBoost aligned closely with actual performance. Findings highlight the non-linear and context-dependent nature of achievement and the value of XAI in educational research. This study uncovers cross-national patterns, informs equity-focused reforms, and supports the development of personalized learning strategies.