Random Vector Functional Link Neural Network based Ensemble Deep Learning

TL;DR

This paper introduces a novel deep learning framework based on randomized neural networks, specifically the deep RVFL and ensemble deep RVFL, which achieve superior performance on benchmark datasets by combining ensemble learning with deep architectures.

Contribution

The paper presents a new deep RVFL network with fixed random hidden layers and a novel ensemble deep RVFL that trains a single model for ensemble benefits, applicable to various RVFL variants.

Findings

Deep RVFL outperforms traditional models on benchmarks.

Ensemble deep RVFL achieves high accuracy with single training.

Framework is versatile with different RVFL variants.

Abstract

In this paper, we propose a deep learning framework based on randomized neural network. In particular, inspired by the principles of Random Vector Functional Link (RVFL) network, we present a deep RVFL network (dRVFL) with stacked layers. The parameters of the hidden layers of the dRVFL are randomly generated within a suitable range and kept fixed while the output weights are computed using the closed form solution as in a standard RVFL network. We also propose an ensemble deep network (edRVFL) that can be regarded as a marriage of ensemble learning with deep learning. Unlike traditional ensembling approaches that require training several models independently from scratch, edRVFL is obtained by training a single dRVFL network once. Both dRVFL and edRVFL frameworks are generic and can be used with any RVFL variant. To illustrate this, we integrate the deep learning networks with a recently proposed sparse-pretrained RVFL (SP-RVFL). Extensive experiments on benchmark datasets from diverse domains show the superior performance of our proposed deep RVFL networks.