Metric as Transform: Exploring beyond Affine Transform for Interpretable Neural Network

TL;DR

This paper investigates the use of metric-based transformations beyond affine transforms in neural networks, demonstrating comparable performance and enhanced interpretability, especially in local neural network models for understanding adversarial examples.

Contribution

It introduces the concept of metric as transform in neural networks, extending beyond affine transformations, and shows its potential for improved interpretability and adversarial example analysis.

Findings

Metrics perform similarly to affine transforms in neural networks.

Metrics can offer better interpretability than affine transforms.

Local metric-based neural networks help understand and reject adversarial examples.

Abstract

Artificial Neural Networks of varying architectures are generally paired with affine transformation at the core. However, we find dot product neurons with global influence less interpretable as compared to local influence of euclidean distance (as used in Radial Basis Function Network). In this work, we explore the generalization of dot product neurons to $l^p$-norm, metrics, and beyond. We find that metrics as transform performs similarly to affine transform when used in MultiLayer Perceptron or Convolutional Neural Network. Moreover, we explore various properties of Metrics, compare it with Affine, and present multiple cases where metrics seem to provide better interpretability. We develop an interpretable local dictionary based Neural Networks and use it to understand and reject adversarial examples.