A Generic Visualization Approach for Convolutional Neural Networks

TL;DR

This paper introduces a novel, architecture-agnostic attention visualization method called L2-CAF for CNNs, applicable to both classification and retrieval networks, without requiring model modifications or fine-tuning.

Contribution

The paper proposes L2-CAF, a new attention visualization technique that works on pre-trained networks for classification and retrieval tasks, outperforming existing methods.

Findings

Achieves state-of-the-art localization results on classification networks.

Significantly improves attention visualization for retrieval networks over Grad-CAM.

Does not require architectural changes or fine-tuning of the original network.

Abstract

Retrieval networks are essential for searching and indexing. Compared to classification networks, attention visualization for retrieval networks is hardly studied. We formulate attention visualization as a constrained optimization problem. We leverage the unit L2-Norm constraint as an attention filter (L2-CAF) to localize attention in both classification and retrieval networks. Unlike recent literature, our approach requires neither architectural changes nor fine-tuning. Thus, a pre-trained network's performance is never undermined L2-CAF is quantitatively evaluated using weakly supervised object localization. State-of-the-art results are achieved on classification networks. For retrieval networks, significant improvement margins are achieved over a Grad-CAM baseline. Qualitative evaluation demonstrates how the L2-CAF visualizes attention per frame for a recurrent retrieval network. Further ablation studies highlight the computational cost of our approach and compare L2-CAF with other feasible alternatives. Code available at https://bit.ly/3iDBLFv