Pushing the Boundaries of Interpretability: Incremental Enhancements to the Explainable Boosting Machine

TL;DR

This paper enhances the Explainable Boosting Machine with hyperparameter tuning, fairness optimization, and self-supervised pre-training, aiming to improve transparency, robustness, and ethical compliance in high-stakes AI applications.

Contribution

It introduces three novel methodologies to improve EBM's performance, fairness, and cold-start capabilities, advancing interpretable machine learning models.

Findings

Marginal ROC AUC improvements from hyperparameter tuning

Subtle shifts in decision-making behavior observed

Enhanced model robustness and fairness demonstrated

Abstract

The widespread adoption of complex machine learning models in high-stakes domains has brought the "black-box" problem to the forefront of responsible AI research. This paper aims at addressing this issue by improving the Explainable Boosting Machine (EBM), a state-of-the-art glassbox model that delivers both high accuracy and complete transparency. The paper outlines three distinct enhancement methodologies: targeted hyperparameter optimization with Bayesian methods, the implementation of a custom multi-objective function for fairness for hyperparameter optimization, and a novel self-supervised pre-training pipeline for cold-start scenarios. All three methodologies are evaluated across standard benchmark datasets, including the Adult Income, Credit Card Fraud Detection, and UCI Heart Disease datasets. The analysis indicates that while the tuning process yielded marginal improvements in the primary ROC AUC metric, it led to a subtle but important shift in the model's decision-making behavior, demonstrating the value of a multi-faceted evaluation beyond a single performance score. This work is positioned as a critical step toward developing machine learning systems that are not only accurate but also robust, equitable, and transparent, meeting the growing demands of regulatory and ethical compliance.