Autoencoder based Randomized Learning of Feedforward Neural Networks for Regression

TL;DR

This paper explores autoencoder-based randomized learning for feedforward neural networks in regression, proposing improvements to control random weights and biases, but finds it less effective and more complex than other methods.

Contribution

It introduces a method to better control autoencoder-generated weights and biases in randomized neural network training for regression tasks.

Findings

Autoencoder-based learning does not outperform competitors in accuracy.

The proposed method is more complex than alternative approaches.

Improvements do not lead to superior performance in regression.

Abstract

Feedforward neural networks are widely used as universal predictive models to fit data distribution. Common gradient-based learning, however, suffers from many drawbacks making the training process ineffective and time-consuming. Alternative randomized learning does not use gradients but selects hidden node parameters randomly. This makes the training process extremely fast. However, the problem in randomized learning is how to determine the random parameters. A recently proposed method uses autoencoders for unsupervised parameter learning. This method showed superior performance on classification tasks. In this work, we apply this method to regression problems, and, finding that it has some drawbacks, we show how to improve it. We propose a learning method of autoencoders that controls the produced random weights. We also propose how to determine the biases of hidden nodes. We empirically compare autoencoder based learning with other randomized learning methods proposed recently for regression and find that despite the proposed improvement of the autoencoder based learning, it does not outperform its competitors in fitting accuracy. Moreover, the method is much more complex than its competitors.