Models That Are Interpretable But Not Transparent

TL;DR

This paper introduces FaithfulDefense, a method for creating interpretable models that provide fully faithful explanations while minimizing the exposure of the model's decision boundary to protect proprietary information.

Contribution

FaithfulDefense offers a novel approach using set cover formulations and submodularity to generate faithful explanations without fully revealing the model's decision boundary.

Findings

FaithfulDefense achieves fully faithful explanations.

The method effectively balances interpretability and model protection.

It employs set cover and submodularity techniques.

Abstract

Faithful explanations are essential for machine learning models in high-stakes applications. Inherently interpretable models are well-suited for these applications because they naturally provide faithful explanations by revealing their decision logic. However, model designers often need to keep these models proprietary to maintain their value. This creates a tension: we need models that are interpretable--allowing human decision-makers to understand and justify predictions, but not transparent, so that the model's decision boundary is not easily replicated by attackers. Shielding the model's decision boundary is particularly challenging alongside the requirement of completely faithful explanations, since such explanations reveal the true logic of the model for an entire subspace around each query point. This work provides an approach, FaithfulDefense, that creates model explanations for logical models that are completely faithful, yet reveal as little as possible about the decision boundary. FaithfulDefense is based on a maximum set cover formulation, and we provide multiple formulations for it, taking advantage of submodularity.