RbX: Region-based explanations of prediction models

TL;DR

RbX is a new model-agnostic method that provides local explanations for prediction models by constructing feature space regions, quantifying feature importance through escape distances, and ensuring relevance detection.

Contribution

The paper introduces RbX, a novel region-based explanation method that guarantees feature relevance detection and is fully model-agnostic, using a convex polytope approximation.

Findings

RbX effectively detects all locally relevant features.

It satisfies a sparsity axiom ensuring irrelevant features are assigned zero importance.

Experiments show RbX outperforms existing explanation methods.

Abstract

We introduce region-based explanations (RbX), a novel, model-agnostic method to generate local explanations of scalar outputs from a black-box prediction model using only query access. RbX is based on a greedy algorithm for building a convex polytope that approximates a region of feature space where model predictions are close to the prediction at some target point. This region is fully specified by the user on the scale of the predictions, rather than on the scale of the features. The geometry of this polytope - specifically the change in each coordinate necessary to escape the polytope - quantifies the local sensitivity of the predictions to each of the features. These "escape distances" can then be standardized to rank the features by local importance. RbX is guaranteed to satisfy a "sparsity axiom," which requires that features which do not enter into the prediction model are assigned zero importance. At the same time, real data examples and synthetic experiments show how RbX can more readily detect all locally relevant features than existing methods.