Convergence of a New Learning Algorithm

TL;DR

This paper analyzes the convergence properties of a new neural network learning algorithm that is equivalent to backpropagation but offers advantages like biological plausibility, providing conditions for convergence and empirical insights.

Contribution

It derives necessary and sufficient conditions for convergence and proposes a measure for the convergence rate of the new algorithm.

Findings

Convergence depends on network parameters such as neuron number and connection density.

The proposed convergence measure effectively captures the rate of learning.

Simulation results validate the theoretical convergence conditions.

Abstract

A new learning algorithm proposed by Brandt and Lin for neural network [1], [2] has been shown to be mathematically equivalent to the conventional back-propagation learning algorithm, but has several advantages over the backpropagation algorithm, including feedback-network-free implementation and biological plausibility. In this paper, we investigate the convergence of the new algorithm. A necessary and sufficient condition for the algorithm to converge is derived. A convergence measure is proposed to measure the convergence rate of the new algorithm. Simulation studies are conducted to investigate the convergence of the algorithm with respect to the number of neurons, the connection distance, the connection density, the ratio of excitatory/inhibitory synapses, the membrane potentials, and the synapse strengths.