Explainable Concept Generation through Vision-Language Preference Learning for Understanding Neural Networks' Internal Representations

TL;DR

This paper introduces a reinforcement learning-based method to automatically generate meaningful visual concepts for neural network explanations, reducing manual effort and improving the discovery of important high-level features.

Contribution

It proposes a novel RL-based approach to optimize vision-language models for automatic concept generation, enhancing explainability of neural networks.

Findings

Efficiently generates diverse meaningful concepts

Reduces manual effort in concept set creation

Improves understanding of neural network internal representations

Abstract

Understanding the inner representation of a neural network helps users improve models. Concept-based methods have become a popular choice for explaining deep neural networks post-hoc because, unlike most other explainable AI techniques, they can be used to test high-level visual "concepts" that are not directly related to feature attributes. For instance, the concept of "stripes" is important to classify an image as a zebra. Concept-based explanation methods, however, require practitioners to guess and manually collect multiple candidate concept image sets, making the process labor-intensive and prone to overlooking important concepts. Addressing this limitation, in this paper, we frame concept image set creation as an image generation problem. However, since naively using a standard generative model does not result in meaningful concepts, we devise a reinforcement learning-based preference optimization (RLPO) algorithm that fine-tunes a vision-language generative model from approximate textual descriptions of concepts. Through a series of experiments, we demonstrate our method's ability to efficiently and reliably articulate diverse concepts that are otherwise challenging to craft manually.