Visualizing the Emergence of Intermediate Visual Patterns in DNNs

TL;DR

This paper introduces a visualization method to analyze how deep neural networks develop and utilize intermediate visual patterns during training, providing insights into their learning process and representation capacity.

Contribution

It presents a novel visualization technique for intermediate-layer patterns in DNNs, enabling quantification of learned discriminative features and analysis of signal-processing behaviors.

Findings

Visualizes the gradual learning of regional patterns in each layer.

Quantifies the number of discriminative visual patterns learned.

Provides insights into adversarial attacks and knowledge distillation behaviors.

Abstract

This paper proposes a method to visualize the discrimination power of intermediate-layer visual patterns encoded by a DNN. Specifically, we visualize (1) how the DNN gradually learns regional visual patterns in each intermediate layer during the training process, and (2) the effects of the DNN using non-discriminative patterns in low layers to construct disciminative patterns in middle/high layers through the forward propagation. Based on our visualization method, we can quantify knowledge points (i.e., the number of discriminative visual patterns) learned by the DNN to evaluate the representation capacity of the DNN. Furthermore, this method also provides new insights into signal-processing behaviors of existing deep-learning techniques, such as adversarial attacks and knowledge distillation.