Glitches in Decision Tree Ensemble Models

TL;DR

This paper introduces the concept of glitches in decision tree ensemble models, formalizes their definition, demonstrates their widespread existence, and develops an MILP-based algorithm to detect them, highlighting potential model unreliability.

Contribution

It formally defines glitches in decision tree models, proves the NP-completeness of detecting them, and presents an effective MILP-based detection algorithm for GBDT models.

Findings

Glitches are widespread in decision tree models.

Detecting glitches is NP-complete for tree ensembles of depth 4.

The proposed MILP algorithm effectively finds glitches in benchmarks.

Abstract

Many critical decision-making tasks are now delegated to machine-learned models, and it is imperative that their decisions are trustworthy and reliable, and their outputs are consistent across similar inputs. We identify a new source of unreliable behaviors-called glitches-which may significantly impair the reliability of AI models having steep decision boundaries. Roughly speaking, glitches are small neighborhoods in the input space where the model's output abruptly oscillates with respect to small changes in the input. We provide a formal definition of glitches, and use well-known models and datasets from the literature to demonstrate that they have widespread existence and argue they usually indicate potential model inconsistencies in the neighborhood of where they are found. We proceed to the algorithmic search of glitches for widely used gradient-boosted decision tree (GBDT) models. We prove that the problem of detecting glitches is NP-complete for tree ensembles, already for trees of depth 4. Our glitch-search algorithm for GBDT models uses an MILP encoding of the problem, and its effectiveness and computational feasibility are demonstrated on a set of widely used GBDT benchmarks taken from the literature.