A Neural Network Training Method Based on Neuron Connection Coefficient Adjustments

TL;DR

This paper introduces a biologically interpretable neural network training method that adjusts neuron connection coefficients via backward signal propagation, avoiding chain rule reliance and improving training robustness.

Contribution

The paper presents a novel training approach based on neuron connection coefficient adjustments that bypasses the chain rule and enhances biological plausibility.

Findings

Achieved promising results on MNIST dataset

Effectively avoids certain local minima during training

Identified limitations and proposed improvements for the neural network architecture

Abstract

In previous studies, we introduced a neural network framework based on symmetric differential equations, along with one of its training methods. In this article, we present another training approach for this neural network. This method leverages backward signal propagation and eliminates reliance on the traditional chain derivative rule, offering a high degree of biological interpretability. Unlike the previously introduced method, this approach does not require adjustments to the fixed points of the differential equations. Instead, it focuses solely on modifying the connection coefficients between neurons, closely resembling the training process of traditional multilayer perceptron (MLP) networks. By adopting a suitable adjustment strategy, this method effectively avoids certain potential local minima. To validate this approach, we tested it on the MNIST dataset and achieved promising results. Through further analysis, we identified certain limitations of the current neural network architecture and proposed measures for improvement.