Neurons Activation Visualization and Information Theoretic Analysis

TL;DR

This paper uses entropy analysis to study neuron activation patterns in deep neural networks, revealing that activation entropy decreases with depth, indicating increased stability and providing insights into network design.

Contribution

It introduces entropy-based analysis of neuron activations in fully connected layers, offering a new perspective on understanding DNN inner workings and layer depth effects.

Findings

Neuron activation entropy decreases with layer depth.

Activation patterns become more stable in deeper layers.

Entropy patterns guide optimal layer number for accuracy.

Abstract

Understanding the inner working mechanism of deep neural networks (DNNs) is essential and important for researchers to design and improve the performance of DNNs. In this work, the entropy analysis is leveraged to study the neurons activation behavior of the fully connected layers of DNNs. The entropy of the activation patterns of each layer can provide a performance metric for the evaluation of the network model accuracy. The study is conducted based on a well trained network model. The activation patterns of shallow and deep layers of the fully connected layers are analyzed by inputting the images of a single class. It is found that for the well trained deep neural networks model, the entropy of the neuron activation pattern is monotonically reduced with the depth of the layers. That is, the neuron activation patterns become more and more stable with the depth of the fully connected layers. The entropy pattern of the fully connected layers can also provide guidelines as to how many fully connected layers are needed to guarantee the accuracy of the model. The study in this work provides a new perspective on the analysis of DNN, which shows some interesting results.