Pulling back error to the hidden-node parameter technology: Single-hidden-layer feedforward network without output weight

TL;DR

This paper proposes a simplified single-hidden-layer neural network architecture that eliminates the need for output weights, achieving high accuracy and significantly faster training times with just one hidden node.

Contribution

It introduces a novel neural network design that removes the output weight, demonstrating universal approximation capability with a single hidden node and faster training.

Findings

Achieves very small output error with only 1 hidden node

Training is several to thousands of times faster than traditional methods

Eliminates the need for output weights in SLFNs

Abstract

According to conventional neural network theories, the feature of single-hidden-layer feedforward neural networks(SLFNs) resorts to parameters of the weighted connections and hidden nodes. SLFNs are universal approximators when at least the parameters of the networks including hidden-node parameter and output weight are exist. Unlike above neural network theories, this paper indicates that in order to let SLFNs work as universal approximators, one may simply calculate the hidden node parameter only and the output weight is not needed at all. In other words, this proposed neural network architecture can be considered as a standard SLFNs with fixing output weight equal to an unit vector. Further more, this paper presents experiments which show that the proposed learning method tends to extremely reduce network output error to a very small number with only 1 hidden node. Simulation results demonstrate that the proposed method can provide several to thousands of times faster than other learning algorithm including BP, SVM/SVR and other ELM methods.