A Neural Network Training Method Based on Distributed PID Control

TL;DR

This paper introduces a novel neural network training method based on distributed PID control and differential equation signal propagation, improving training speed and accuracy while enhancing biological interpretability.

Contribution

It presents a new training approach using differential equations and distributed PID control within a symmetric neural network framework, offering faster and more accurate training.

Findings

Faster training speeds achieved on MNIST dataset

Improved accuracy over traditional methods

Enhanced biological interpretability of neural training

Abstract

In the previous article, we introduced a neural network framework based on symmetric differential equations. This novel framework exhibits complete symmetry, endowing it with perfect mathematical properties. While we have examined some of the system's mathematical characteristics, a detailed discussion of the network training methodology has not yet been presented. Drawing on the principles of the traditional backpropagation algorithm, this study proposes an alternative training approach that utilizes differential equation signal propagation instead of chain rule derivation. This approach not only preserves the effectiveness of training but also offers enhanced biological interpretability. The foundation of this methodology lies in the system's reversibility, which stems from its inherent symmetry,a key aspect of our research. However, this method alone is insufficient for effective neural network training. To address this, we further introduce a distributed Proportional-Integral-Derivative (PID) control approach, emphasizing its implementation within a closed system. By incorporating this method, we achieved both faster training speeds and improved accuracy. This approach not only offers novel insights into neural network training but also extends the scope of research into control methodologies. To validate its effectiveness, we apply this method to the MNIST dataset, demonstrating its practical utility.