Neural networks with dynamical coefficients and adjustable connections on the basis of integrated backpropagation

TL;DR

This paper introduces a novel neural network framework where neurons autonomously update their weights using internal backpropagation-based rules, enabling more integrated and potentially efficient learning processes.

Contribution

It presents a new neuron model incorporating internal backpropagation error estimates, allowing weight updates and error exchange without external training procedures.

Findings

Neurons can update weights internally using backpropagation error estimates.

The framework applies to recurrent networks, including LSTM and CNNs.

Alternative descriptions of standard neural networks within this formalism are provided.

Abstract

We consider artificial neurons which will update their weight coefficients with an internal rule based on backpropagation, rather than using it as an external training procedure. To achieve this we include the backpropagation error estimate as a separate entity in all the neuron models and perform its exchange along the synaptic connections. In addition to this we add some special type of neurons with reference inputs, which will serve as a base source of error estimates for the whole network. Finally, we introduce a training control signal for all the neurons, which can enable the correction of weights and the exchange of error estimates. For recurrent neural networks we also demonstrate how to integrate backpropagation through time into their formalism with the help of some stack memory for reference inputs and external data inputs of neurons. Also, for widely used neural networks, such as long short-term memory, radial basis function networks, multilayer perceptrons and convolutional neural networks, we demonstrate their alternative description within the framework of our new formalism.