Interpretation of ResNet by Visualization of Preferred Stimulus in Receptive Fields

TL;DR

This paper investigates the biological interpretability of ResNet by visualizing its receptive fields, revealing orientation and color selectivity, and analyzing the influence of inactive neurons on classification performance.

Contribution

It provides the first biological interpretation of ResNet's receptive fields, linking neural features to visual cortex properties and neuron activity.

Findings

ResNet has orientation selective neurons.

ResNet contains double opponent color neurons.

Inactive neurons in the first layer influence classification.

Abstract

One of the methods used in image recognition is the Deep Convolutional Neural Network (DCNN). DCNN is a model in which the expressive power of features is greatly improved by deepening the hidden layer of CNN. The architecture of CNNs is determined based on a model of the visual cortex of mammals. There is a model called Residual Network (ResNet) that has a skip connection. ResNet is an advanced model in terms of the learning method, but it has not been interpreted from a biological viewpoint. In this research, we investigate the receptive fields of a ResNet on the classification task in ImageNet. We find that ResNet has orientation selective neurons and double opponent color neurons. In addition, we suggest that some inactive neurons in the first layer of ResNet affect the classification task.