Optimal probabilistic feature shifts for reclassification in tree ensembles

TL;DR

This paper introduces a mathematical optimization approach to perturb features of observations in tree ensemble classifiers, aiming to reclassify them into desired classes with maximized probability, while also ranking feature importance.

Contribution

It presents a novel optimization-based method for feature perturbation in tree ensembles, enabling targeted reclassification and feature importance ranking.

Findings

Method effectively reclassifies observations into desired classes.

Validates approach on real dataset.

Provides feature importance ranking in tree ensembles.

Abstract

In this paper we provide a novel mathematical optimization based methodology to perturb the features of a given observation to be re-classified, by a tree ensemble classification rule, to a certain desired class. The method is based on these facts: the most viable changes for an observation to reach the desired class do not always coincide with the closest distance point (in the feature space) of the target class; individuals put effort on a few number of features to reach the desired class; and each individual is endowed with a probability to change each of its features to a given value, which determines the overall probability of changing to the target class. Putting all together, we provide different methods to find the features where the individuals must exert effort to maximize the probability to reach the target class. Our method also allows us to rank the most important features in the tree-ensemble. The proposed methodology is tested on a real dataset, validating the proposal.