Restyling Unsupervised Concept Based Interpretable Networks with Generative Models

TL;DR

This paper introduces a novel approach that maps high-level concepts in interpretable neural networks to the latent space of pretrained generative models, enabling better visualization, interpretation, and training efficiency for large-scale image recognition tasks.

Contribution

The method leverages pretrained generative models to improve visualization, interpretability, and training efficiency of unsupervised concept-based neural networks.

Findings

High-quality visualization of concepts achieved

Improved interpretability and fidelity of learned concepts

Efficient training through leveraging pretrained generative models

Abstract

Developing inherently interpretable models for prediction has gained prominence in recent years. A subclass of these models, wherein the interpretable network relies on learning high-level concepts, are valued because of closeness of concept representations to human communication. However, the visualization and understanding of the learnt unsupervised dictionary of concepts encounters major limitations, especially for large-scale images. We propose here a novel method that relies on mapping the concept features to the latent space of a pretrained generative model. The use of a generative model enables high quality visualization, and lays out an intuitive and interactive procedure for better interpretation of the learnt concepts by imputing concept activations and visualizing generated modifications. Furthermore, leveraging pretrained generative models has the additional advantage of making the training of the system more efficient. We quantitatively ascertain the efficacy of our method in terms of accuracy of the interpretable prediction network, fidelity of reconstruction, as well as faithfulness and consistency of learnt concepts. The experiments are conducted on multiple image recognition benchmarks for large-scale images. Project page available at https://jayneelparekh.github.io/VisCoIN_project_page/