Learning Invariances for Interpretability using Supervised VAE

TL;DR

This paper introduces a supervised variational auto-encoder framework to learn and interpret model invariances, enabling better understanding of how complex models solve problems by analyzing invariant transformations.

Contribution

It proposes a novel supervised VAE approach that isolates nuisance parameters to reveal model invariances, enhancing interpretability of supervised models.

Findings

Successfully generates invariant transformations that do not change classification.

Improves understanding of model decision processes through invariance analysis.

Enhances classification and sample generation with invariance-aware models.

Abstract

We propose to learn model invariances as a means of interpreting a model. This is motivated by a reverse engineering principle. If we understand a problem, we may introduce inductive biases in our model in the form of invariances. Conversely, when interpreting a complex supervised model, we can study its invariances to understand how that model solves a problem. To this end we propose a supervised form of variational auto-encoders (VAEs). Crucially, only a subset of the dimensions in the latent space contributes to the supervised task, allowing the remaining dimensions to act as nuisance parameters. By sampling solely the nuisance dimensions, we are able to generate samples that have undergone transformations that leave the classification unchanged, revealing the invariances of the model. Our experimental results show the capability of our proposed model both in terms of classification, and generation of invariantly transformed samples. Finally we show how combining our model with feature attribution methods it is possible to reach a more fine-grained understanding about the decision process of the model.