Understanding Multimodal Deep Neural Networks: A Concept Selection View

TL;DR

This paper introduces a two-stage concept selection model for understanding multimodal neural networks like CLIP, extracting core concepts without human labels, and demonstrating interpretability and comparable performance.

Contribution

A novel two-stage concept selection approach that identifies core concepts in multimodal models without relying on human-labeled data.

Findings

Achieves comparable performance to black-box models.

Concepts discovered are interpretable and comprehensible.

Effective in mining core concepts without human priors.

Abstract

The multimodal deep neural networks, represented by CLIP, have generated rich downstream applications owing to their excellent performance, thus making understanding the decision-making process of CLIP an essential research topic. Due to the complex structure and the massive pre-training data, it is often regarded as a black-box model that is too difficult to understand and interpret. Concept-based models map the black-box visual representations extracted by deep neural networks onto a set of human-understandable concepts and use the concepts to make predictions, enhancing the transparency of the decision-making process. However, these methods involve the datasets labeled with fine-grained attributes by expert knowledge, which incur high costs and introduce excessive human prior knowledge and bias. In this paper, we observe the long-tail distribution of concepts, based on which we propose a two-stage Concept Selection Model (CSM) to mine core concepts without introducing any human priors. The concept greedy rough selection algorithm is applied to extract head concepts, and then the concept mask fine selection method performs the extraction of core concepts. Experiments show that our approach achieves comparable performance to end-to-end black-box models, and human evaluation demonstrates that the concepts discovered by our method are interpretable and comprehensible for humans.