Achieving interpretable machine learning by functional decomposition of black-box models into explainable predictor effects

TL;DR

This paper introduces a novel method for interpreting black-box machine learning models by decomposing their predictions into simpler, explainable functions using stacked orthogonality, improving interpretability without sacrificing accuracy.

Contribution

The paper presents a new functional decomposition approach for black-box models using stacked orthogonality, enabling clearer insights into feature effects and interactions.

Findings

Method effectively isolates main effects and interactions.

Approach is robust against extrapolation and hidden interactions.

Algorithm combines neural additive modeling with orthogonalization.

Abstract

Machine learning (ML) has seen significant growth in both popularity and importance. The high prediction accuracy of ML models is often achieved through complex black-box architectures that are difficult to interpret. This interpretability problem has been hindering the use of ML in fields like medicine, ecology and insurance, where an understanding of the inner workings of the model is paramount to ensure user acceptance and fairness. The need for interpretable ML models has boosted research in the field of interpretable machine learning (IML). Here we propose a novel approach for the functional decomposition of black-box predictions, which is considered a core concept of IML. The idea of our method is to replace the prediction function by a surrogate model consisting of simpler subfunctions. Similar to additive regression models, these functions provide insights into the direction and strength of the main feature contributions and their interactions. Our method is based on a novel concept termed stacked orthogonality, which ensures that the main effects capture as much functional behavior as possible and do not contain information explained by higher-order interactions. Unlike earlier functional IML approaches, it is neither affected by extrapolation nor by hidden feature interactions. To compute the subfunctions, we propose an algorithm based on neural additive modeling and an efficient post-hoc orthogonalization procedure.