Energy Functional and Fixed Points of a Neural Networks

TL;DR

This paper investigates the relationship between fixed points and local minima in neural networks' energy functionals, revealing conditions under which they differ and proposing a method to eliminate fictitious fixed points, especially with projection matrices.

Contribution

It introduces a simple method to exclude fictitious fixed points with high energies, particularly effective when using projection connection matrices.

Findings

Fixed points are not always local minima, depending on the connection matrix.

A method to cut off high-energy fictitious fixed points was developed.

The method is especially effective with projection matrices.

Abstract

It turned out that the set of the fixed points is not necessarily the same as the set of the local minima of the energy functional. It depends on the diagonal elements of the connection matrix. The simple method which allows to cut off fictitious fixed points with high energies was found out. Especially the method is effective if the connection matrix is the projection one.