Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps

TL;DR

This paper explores visualization techniques for deep convolutional networks, including gradient-based class visualization and saliency maps, demonstrating their use in object segmentation and connecting them to deconvolutional networks.

Contribution

It introduces gradient-based visualization methods for ConvNets, including class visualization and saliency maps, and links these techniques to deconvolutional networks.

Findings

Saliency maps enable weakly supervised object segmentation.

Gradient-based visualizations reveal class-specific features.

Connections established between visualization methods and deconvolutional networks.

Abstract

This paper addresses the visualisation of image classification models, learnt using deep Convolutional Networks (ConvNets). We consider two visualisation techniques, based on computing the gradient of the class score with respect to the input image. The first one generates an image, which maximises the class score [Erhan et al., 2009], thus visualising the notion of the class, captured by a ConvNet. The second technique computes a class saliency map, specific to a given image and class. We show that such maps can be employed for weakly supervised object segmentation using classification ConvNets. Finally, we establish the connection between the gradient-based ConvNet visualisation methods and deconvolutional networks [Zeiler et al., 2013].