Modelling higher education dropouts using sparse and interpretable post-clustering logistic regression

TL;DR

This paper introduces a sparse, interpretable logistic regression model with clustering to identify and characterize student subgroups at risk of dropout, enhancing understanding and policy relevance.

Contribution

It extends traditional logistic regression by integrating clustering and sparsity, enabling unsupervised subgroup identification and interpretability in dropout analysis.

Findings

Effective identification of student subgroups at risk

Enhanced interpretability through sparsity and clustering

Successful application to Italian university data

Abstract

Higher education dropout constitutes a critical challenge for tertiary education systems worldwide. While machine learning techniques can achieve high predictive accuracy on selected datasets, their adoption by policymakers remains limited and unsatisfactory, particularly when the objective is the unsupervised identification and characterization of student subgroups at elevated risk of dropout. The model introduced in this paper is a specialized form of logistic regression, specifically adapted to the context of university dropout analysis. Logistic regression continues to serve as a foundational tool among reliable statistical models, primarily due to the ease with which its parameters can be interpreted in terms of odds ratios. Our approach significantly extends this framework by incorporating heterogeneity within the student population. This is achieved through the application of a preliminary clustering algorithm that identifies latent subgroups, each characterized by distinct dropout propensities, which are then modeled via cluster-specific effects. We provide a detailed interpretation of the model parameters within this extended framework and enhance interpretability by imposing sparsity through a tailored variant of the LASSO algorithm. To demonstrate the practical applicability of the proposed methodology, we present an extensive case study based on the Italian university system, in which all the developed tools are systematically applied