ELUDE: Generating interpretable explanations via a decomposition into labelled and unlabelled features

TL;DR

ELUDE is a novel explanation framework that decomposes neural network predictions into interpretable semantic attributes and a small set of uninterpretable features, providing deeper insights into model behavior.

Contribution

The paper introduces ELUDE, a new method that combines labeled and unlabeled features to explain complex models, extending beyond purely semantic attribute explanations.

Findings

ELUDE effectively decomposes predictions into explainable and unexplained parts.

Unlabeled features generalize across models trained on the same data.

ELUDE offers additional insights compared to existing attribute-based explanation methods.

Abstract

Deep learning models have achieved remarkable success in different areas of machine learning over the past decade; however, the size and complexity of these models make them difficult to understand. In an effort to make them more interpretable, several recent works focus on explaining parts of a deep neural network through human-interpretable, semantic attributes. However, it may be impossible to completely explain complex models using only semantic attributes. In this work, we propose to augment these attributes with a small set of uninterpretable features. Specifically, we develop a novel explanation framework ELUDE (Explanation via Labelled and Unlabelled DEcomposition) that decomposes a model's prediction into two parts: one that is explainable through a linear combination of the semantic attributes, and another that is dependent on the set of uninterpretable features. By identifying the latter, we are able to analyze the "unexplained" portion of the model, obtaining insights into the information used by the model. We show that the set of unlabelled features can generalize to multiple models trained with the same feature space and compare our work to two popular attribute-oriented methods, Interpretable Basis Decomposition and Concept Bottleneck, and discuss the additional insights ELUDE provides.