Interpretable Machine Learning with an Ensemble of Gradient Boosting Machines

TL;DR

This paper introduces an interpretable ensemble method combining gradient boosting machines for local and global model explanations, offering explicit feature weights and simple training compared to neural additive models.

Contribution

It presents a novel ensemble approach using GBMs for interpretability, with explicit feature weights and parallel training, improving interpretability over neural additive models.

Findings

Effective in local and global interpretation

Works well on synthetic and real datasets

Provides explicit feature weights

Abstract

A method for the local and global interpretation of a black-box model on the basis of the well-known generalized additive models is proposed. It can be viewed as an extension or a modification of the algorithm using the neural additive model. The method is based on using an ensemble of gradient boosting machines (GBMs) such that each GBM is learned on a single feature and produces a shape function of the feature. The ensemble is composed as a weighted sum of separate GBMs resulting a weighted sum of shape functions which form the generalized additive model. GBMs are built in parallel using randomized decision trees of depth 1, which provide a very simple architecture. Weights of GBMs as well as features are computed in each iteration of boosting by using the Lasso method and then updated by means of a specific smoothing procedure. In contrast to the neural additive model, the method provides weights of features in the explicit form, and it is simply trained. A lot of numerical experiments with an algorithm implementing the proposed method on synthetic and real datasets demonstrate its efficiency and properties for local and global interpretation.