Comparative performance of some popular ANN algorithms on benchmark and function approximation problems

TL;DR

This paper compares various popular ANN algorithms on benchmark classification and function approximation problems, highlighting their relative performance and suggesting certain functions as new benchmarks.

Contribution

It provides a comparative analysis of multiple ANN algorithms on standard benchmarks and regression problems, identifying the best performers for each task.

Findings

Levenberg-Marquardt yields lowest RMS error for parity and spiral problems.

Higher Order Neurons excel on IRIS data.

Neuro Fuzzy performs best on XOR problem.

Abstract

We report an inter-comparison of some popular algorithms within the artificial neural network domain (viz., Local search algorithms, global search algorithms, higher order algorithms and the hybrid algorithms) by applying them to the standard benchmarking problems like the IRIS data, XOR/N-Bit parity and Two Spiral. Apart from giving a brief description of these algorithms, the results obtained for the above benchmark problems are presented in the paper. The results suggest that while Levenberg-Marquardt algorithm yields the lowest RMS error for the N-bit Parity and the Two Spiral problems, Higher Order Neurons algorithm gives the best results for the IRIS data problem. The best results for the XOR problem are obtained with the Neuro Fuzzy algorithm. The above algorithms were also applied for solving several regression problems such as cos(x) and a few special functions like the Gamma function, the complimentary Error function and the upper tail cumulative $\chi^2$-distribution function. The results of these regression problems indicate that, among all the ANN algorithms used in the present study, Levenberg-Marquardt algorithm yields the best results. Keeping in view the highly non-linear behaviour and the wide dynamic range of these functions, it is suggested that these functions can be also considered as standard benchmark problems for function approximation using artificial neural networks.