Stacked Autoencoder Based Deep Random Vector Functional Link Neural Network for Classification

TL;DR

This paper introduces a novel deep RVFL neural network architecture using stacked autoencoders with direct feature reuse and denoising criteria, leading to improved classification performance and faster generalization.

Contribution

It proposes deep RVFL variants with direct connections and denoising autoencoders, enhancing regularization, reducing complexity, and improving classification accuracy over existing deep networks.

Findings

Achieved better classification accuracy on multiple datasets.

Demonstrated faster convergence and generalization.

Outperformed state-of-the-art deep neural networks.

Abstract

Extreme learning machine (ELM), which can be viewed as a variant of Random Vector Functional Link (RVFL) network without the input-output direct connections, has been extensively used to create multi-layer (deep) neural networks. Such networks employ randomization based autoencoders (AE) for unsupervised feature extraction followed by an ELM classifier for final decision making. Each randomization based AE acts as an independent feature extractor and a deep network is obtained by stacking several such AEs. Inspired by the better performance of RVFL over ELM, in this paper, we propose several deep RVFL variants by utilizing the framework of stacked autoencoders. Specifically, we introduce direct connections (feature reuse) from preceding layers to the fore layers of the network as in the original RVFL network. Such connections help to regularize the randomization and also reduce the model complexity. Furthermore, we also introduce denoising criterion, recovering clean inputs from their corrupted versions, in the autoencoders to achieve better higher level representations than the ordinary autoencoders. Extensive experiments on several classification datasets show that our proposed deep networks achieve overall better and faster generalization than the other relevant state-of-the-art deep neural networks.