An unfeasability view of neural network learning

TL;DR

This paper demonstrates that under certain conditions, there are no perfectly continuous differentiable learning algorithms for multilayer neural networks with specific activation functions when the data set exceeds the number of parameters.

Contribution

It provides a theoretical impossibility result showing the non-existence of certain ideal learning algorithms for neural networks with logistic, tanh, or sin activations.

Findings

No continuously differentiable perfect learning algorithms exist under specified conditions.

The result applies when data set length exceeds number of parameters.

The proof covers logistic, tanh, and sin activation functions.

Abstract

We define the notion of a continuously differentiable perfect learning algorithm for multilayer neural network architectures and show that such algorithms don't exist provided that the length of the data set exceeds the number of involved parameters and the activation functions are logistic, tanh or sin.