Representation Understanding via Activation Maximization

TL;DR

This paper introduces a unified framework for visualizing and understanding deep neural network features across CNNs and Vision Transformers, extending to intermediate layers and exploring adversarial example generation for interpretability.

Contribution

It proposes a comprehensive feature visualization method applicable to various network architectures and layers, enhancing interpretability and vulnerability analysis.

Findings

Effective visualization of intermediate and final layer features.

Demonstrated generalizability to CNNs and ViTs.

Revealed potential for adversarial example generation.

Abstract

Understanding internal feature representations of deep neural networks (DNNs) is a fundamental step toward model interpretability. Inspired by neuroscience methods that probe biological neurons using visual stimuli, recent deep learning studies have employed Activation Maximization (AM) to synthesize inputs that elicit strong responses from artificial neurons. In this work, we propose a unified feature visualization framework applicable to both Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs). Unlike prior efforts that predominantly focus on the last output-layer neurons in CNNs, we extend feature visualization to intermediate layers as well, offering deeper insights into the hierarchical structure of learned feature representations. Furthermore, we investigate how activation maximization can be leveraged to generate adversarial examples, revealing potential vulnerabilities and decision boundaries of DNNs. Our experiments demonstrate the effectiveness of our approach in both traditional CNNs and modern ViT, highlighting its generalizability and interpretive value.