Comparing Deep Neural Nets with UMAP Tour

TL;DR

This paper introduces UMAP Tour, a visualization tool for inspecting and comparing neural network internal representations, revealing learned concepts and differences between models like GoogLeNet and ResNet.

Contribution

The paper presents a novel visualization method and a new similarity measure for neural network layers, enabling detailed comparison of learned concepts across models.

Findings

Identified distinct concepts learned by different models

Revealed dissimilarities between GoogLeNet and ResNet

Provided a new approach for neural network interpretability

Abstract

Neural networks should be interpretable to humans. In particular, there is a growing interest in concepts learned in a layer and similarity between layers. In this work, a tool, UMAP Tour, is built to visually inspect and compare internal behavior of real-world neural network models using well-aligned, instance-level representations. The method used in the visualization also implies a new similarity measure between neural network layers. Using the visual tool and the similarity measure, we find concepts learned in state-of-the-art models and dissimilarities between them, such as GoogLeNet and ResNet.