A Method of Generating Random Weights and Biases in Feedforward Neural Networks with Random Hidden Nodes

TL;DR

This paper proposes a data-dependent method for generating random weights and biases in neural networks with hidden nodes, improving their approximation capabilities and control over generalization.

Contribution

It introduces a novel approach to generate random parameters based on data range and activation function, enhancing neural network performance.

Findings

Improved approximation accuracy in experiments

Enhanced control over model generalization

Promising results demonstrated in multiple tests

Abstract

Neural networks with random hidden nodes have gained increasing interest from researchers and practical applications. This is due to their unique features such as very fast training and universal approximation property. In these networks the weights and biases of hidden nodes determining the nonlinear feature mapping are set randomly and are not learned. Appropriate selection of the intervals from which weights and biases are selected is extremely important. This topic has not yet been sufficiently explored in the literature. In this work a method of generating random weights and biases is proposed. This method generates the parameters of the hidden nodes in such a way that nonlinear fragments of the activation functions are located in the input space regions with data and can be used to construct the surface approximating a nonlinear target function. The weights and biases are dependent on the input data range and activation function type. The proposed methods allows us to control the generalization degree of the model. These all lead to improvement in approximation performance of the network. Several experiments show very promising results.