PCA- and SVM-Grad-CAM for Convolutional Neural Networks: Closed-form Jacobian Expression

TL;DR

This paper introduces PCA-Grad-CAM and SVM-Grad-CAM, novel visualization techniques for CNNs with PCA and SVM layers, by deriving exact closed-form Jacobians to interpret attention regions in these layers.

Contribution

It provides the first analytical derivation of closed-form Jacobians for PCA and SVM layers in CNNs, enabling visualization of attention regions in these non-standard layers.

Findings

Successful visualization of attention regions in PCA and SVM layers.

Improved interpretability of CNNs with PCA and SVM components.

Application to major datasets demonstrating effectiveness.

Abstract

Convolutional Neural Networks (CNNs) are an effective approach for classification tasks, particularly when the training dataset is large. Although CNNs have long been considered a black-box classification method, they can be used as a white-box method through visualization techniques such as Grad-CAM. When training samples are limited, incorporating a Principal Component Analysis (PCA) layer and/or a Support Vector Machine (SVM) classifier into a CNN can effectively improve classification performance. However, traditional Grad-CAM cannot be directly applied to PCA and/or SVM layers. It is important to generate attention regions for PCA and/or SVM layers in CNNs to facilitate the development of white-box methods. Therefore, we propose ``PCA-Grad-CAM'', a method for visualizing attention regions in PCA feature vectors, and ``SVM-Grad-CAM'', a method for visualizing attention regions in an SVM classifier layer. To complete our methods analytically, it is necessary to solve the closed-form Jacobian consisting of partial derivatives from the last convolutional layer to the PCA and/or SVM layers. In this paper, we present the exact closed-form Jacobian and the visualization results of our methods applied to several major datasets.