# Neural network identifiability for a family of sigmoidal nonlinearities

## TL;DR

This paper investigates when the input-output map of a neural network with various nonlinearities uniquely determines its architecture and parameters, providing minimal conditions for identifiability across diverse network structures.

## Contribution

It derives necessary genericity conditions for neural network identifiability of arbitrary depth and connectivity, and constructs a broad family of nonlinearities satisfying these conditions.

## Key findings

- Identifiability conditions are established for networks of any depth and connectivity.
- A large family of nonlinearities is constructed that meets the minimal genericity conditions.
- The family of nonlinearities can approximate many common nonlinear functions arbitrarily well.

## Abstract

This paper addresses the following question of neural network identifiability: Does the input-output map realized by a feed-forward neural network with respect to a given nonlinearity uniquely specify the network architecture, weights, and biases? Existing literature on the subject Sussman 1992, Albertini, Sontag et al. 1993, Fefferman 1994 suggests that the answer should be yes, up to certain symmetries induced by the nonlinearity, and provided the networks under consideration satisfy certain "genericity conditions". The results in Sussman 1992 and Albertini, Sontag et al. 1993 apply to networks with a single hidden layer and in Fefferman 1994 the networks need to be fully connected. In an effort to answer the identifiability question in greater generality, we derive necessary genericity conditions for the identifiability of neural networks of arbitrary depth and connectivity with an arbitrary nonlinearity. Moreover, we construct a family of nonlinearities for which these genericity conditions are minimal, i.e., both necessary and sufficient. This family is large enough to approximate many commonly encountered nonlinearities to within arbitrary precision in the uniform norm.