Hamming Similarity and Graph Laplacians for Class Partitioning and Adversarial Image Detection

TL;DR

This paper explores how ReLU activation patterns encoded as bit vectors can be used to understand neural network behavior, improve class separation, and detect adversarial images with high accuracy.

Contribution

It introduces a novel approach using Hamming similarity and graph Laplacians on activation bit vectors for class partitioning and adversarial detection.

Findings

ReLU bit vectors achieve over 95% class separation accuracy.

Bit vectors effectively distinguish adversarial from non-adversarial images.

Fiedler partitioning improves interpretability of neural network representations.

Abstract

Researchers typically investigate neural network representations by examining activation outputs for one or more layers of a network. Here, we investigate the potential for ReLU activation patterns (encoded as bit vectors) to aid in understanding and interpreting the behavior of neural networks. We utilize Representational Dissimilarity Matrices (RDMs) to investigate the coherence of data within the embedding spaces of a deep neural network. From each layer of a network, we extract and utilize bit vectors to construct similarity scores between images. From these similarity scores, we build a similarity matrix for a collection of images drawn from 2 classes. We then apply Fiedler partitioning to the associated Laplacian matrix to separate the classes. Our results indicate, through bit vector representations, that the network continues to refine class detectability with the last ReLU layer achieving better than 95\% separation accuracy. Additionally, we demonstrate that bit vectors aid in adversarial image detection, again achieving over 95\% accuracy in separating adversarial and non-adversarial images using a simple classifier.