Feature Activation Map: Visual Explanation of Deep Learning Models for Image Classification

TL;DR

This paper introduces Feature Activation Map (FAM), a new visualization method that explains deep learning models without fully-connected layers, applicable to various image classification tasks like few-shot learning and image retrieval.

Contribution

FAM is a novel post-hoc interpretation tool that generates visual explanations for models lacking fully-connected layers, extending interpretability to a broader range of deep learning architectures.

Findings

FAM effectively visualizes models in few-shot learning scenarios.

FAM provides clear explanations in contrastive learning and image retrieval tasks.

Experimental results show FAM outperforms existing CAM-based methods in interpretability.

Abstract

Decisions made by convolutional neural networks(CNN) can be understood and explained by visualizing discriminative regions on images. To this end, Class Activation Map (CAM) based methods were proposed as powerful interpretation tools, making the prediction of deep learning models more explainable, transparent, and trustworthy. However, all the CAM-based methods (e.g., CAM, Grad-CAM, and Relevance-CAM) can only be used for interpreting CNN models with fully-connected (FC) layers as a classifier. It is worth noting that many deep learning models classify images without FC layers, e.g., few-shot learning image classification, contrastive learning image classification, and image retrieval tasks. In this work, a post-hoc interpretation tool named feature activation map (FAM) is proposed, which can interpret deep learning models without FC layers as a classifier. In the proposed FAM algorithm, the channel-wise contribution weights are derived from the similarity scores between two image embeddings. The activation maps are linearly combined with the corresponding normalized contribution weights, forming the explanation map for visualization. The quantitative and qualitative experiments conducted on ten deep learning models for few-shot image classification, contrastive learning image classification and image retrieval tasks demonstrate the effectiveness of the proposed FAM algorithm.