Inverting the Feature Visualization Process for Feedforward Neural Networks

TL;DR

This paper explores the invertibility of feature visualization in neural networks by deriving a closed-form solution to optimize feature objectives, offering new insights into network sensitivity beyond activation patterns.

Contribution

It introduces a method to invert feature visualization by computing a closed-form solution for optimal feature objectives, enhancing understanding of neural network sensitivity.

Findings

Closed-form solution for feature objective optimization

Enhanced interpretation of network sensitivity

Alternative perspective on feature visualization

Abstract

This work sheds light on the invertibility of feature visualization in neural networks. Since the input that is generated by feature visualization using activation maximization does, in general, not yield the feature objective it was optimized for, we investigate optimizing for the feature objective that yields this input. Given the objective function used in activation maximization that measures how closely a given input resembles the feature objective, we exploit that the gradient of this function w.r.t. inputs is---up to a scaling factor---linear in the objective. This observation is used to find the optimal feature objective via computing a closed form solution that minimizes the gradient. By means of Inverse Feature Visualization, we intend to provide an alternative view on a networks sensitivity to certain inputs that considers feature objectives rather than activations.