GENNI: Visualising the Geometry of Equivalences for Neural Network Identifiability

TL;DR

This paper introduces GENNI, an efficient algorithm to visualize symmetries and equivalence classes in neural network parameters, aiding understanding of model identifiability and its implications for optimization and generalization.

Contribution

GENNI is a novel method that efficiently identifies and visualizes parameter equivalences in neural networks, enhancing analysis of model symmetries and identifiability.

Findings

GENNI successfully visualizes symmetry subspaces in neural networks.

The method reveals the structure of parameter equivalence classes.

Applications include improved understanding of optimization landscapes.

Abstract

We propose an efficient algorithm to visualise symmetries in neural networks. Typically, models are defined with respect to a parameter space, where non-equal parameters can produce the same input-output map. Our proposed method, GENNI, allows us to efficiently identify parameters that are functionally equivalent and then visualise the subspace of the resulting equivalence class. By doing so, we are now able to better explore questions surrounding identifiability, with applications to optimisation and generalizability, for commonly used or newly developed neural network architectures.