An Improved Gauss-Newtons Method based Back-propagation Algorithm for Fast Convergence

TL;DR

This paper introduces an enhanced back-propagation algorithm utilizing Gauss-Newton optimization to achieve faster convergence in training multilayer neural networks, with analysis of memory requirements and empirical testing on various datasets.

Contribution

It presents a novel back-propagation algorithm based on Gauss-Newton method, improving convergence speed over traditional steepest descent approaches.

Findings

Faster convergence observed during training

Reduced memory requirements for the algorithm

Effective across multiple datasets

Abstract

The present work deals with an improved back-propagation algorithm based on Gauss-Newton numerical optimization method for fast convergence. The steepest descent method is used for the back-propagation. The algorithm is tested using various datasets and compared with the steepest descent back-propagation algorithm. In the system, optimization is carried out using multilayer neural network. The efficacy of the proposed method is observed during the training period as it converges quickly for the dataset used in test. The requirement of memory for computing the steps of algorithm is also analyzed.