Navigating the Latent Space Dynamics of Neural Models

TL;DR

This paper introduces a novel perspective on neural networks as dynamical systems on latent spaces, enabling new analysis tools for understanding model behavior, generalization, and out-of-distribution detection.

Contribution

It proposes a method to interpret autoencoders as vector fields on latent manifolds, providing insights into model properties and data representations without additional training.

Findings

Latent vector fields can reveal attractor points formed during training.

The approach helps analyze generalization and memorization regimes.

It enables detection of out-of-distribution samples through trajectory analysis.

Abstract

Neural networks transform high-dimensional data into compact, structured representations, often modeled as elements of a lower dimensional latent space. In this paper, we present an alternative interpretation of neural models as dynamical systems acting on the latent manifold. Specifically, we show that autoencoder models implicitly define a latent vector field on the manifold, derived by iteratively applying the encoding-decoding map, without any additional training. We observe that standard training procedures introduce inductive biases that lead to the emergence of attractor points within this vector field. Drawing on this insight, we propose to leverage the vector field as a representation for the network, providing a novel tool to analyze the properties of the model and the data. This representation enables to: (i) analyze the generalization and memorization regimes of neural models, even throughout training; (ii) extract prior knowledge encoded in the network's parameters from the attractors, without requiring any input data; (iii) identify out-of-distribution samples from their trajectories in the vector field. We further validate our approach on vision foundation models, showcasing the applicability and effectiveness of our method in real-world scenarios.