ChannelExplorer: Exploring Class Separability Through Activation Channel Visualization

TL;DR

ChannelExplorer is an interactive visualization tool that helps researchers understand how different activation channels in various neural network architectures contribute to class separability, aiding interpretability of complex models.

Contribution

This paper introduces ChannelExplorer, a novel visual analytics tool for analyzing activation channels and class separability across diverse neural network architectures.

Findings

Effectively visualizes class confusion and separability.

Identifies mislabeled images and activation contributions.

Supports multiple model architectures including CNNs and GANs.

Abstract

Deep neural networks (DNNs) achieve state-of-the-art performance in many vision tasks, yet understanding their internal behavior remains challenging, particularly how different layers and activation channels contribute to class separability. We introduce ChannelExplorer, an interactive visual analytics tool for analyzing image-based outputs across model layers, emphasizing data-driven insights over architecture analysis for exploring class separability. ChannelExplorer summarizes activations across layers and visualizes them using three primary coordinated views: a Scatterplot View to reveal inter- and intra-class confusion, a Jaccard Similarity View to quantify activation overlap, and a Heatmap View to inspect activation channel patterns. Our technique supports diverse model architectures, including CNNs, GANs, ResNet and Stable Diffusion models. We demonstrate the capabilities of ChannelExplorer through four use-case scenarios: (1) generating class hierarchy in ImageNet, (2) finding mislabeled images, (3) identifying activation channel contributions, and(4) locating latent states' position in Stable Diffusion model. Finally, we evaluate the tool with expert users.