A Theoretical Study of The Relationship Between Whole An ELM Network and Its Subnetworks

TL;DR

This paper presents a theoretical analysis of Extreme Learning Machine (ELM) networks and their subnetworks, proving a recursive optimal solution construction, and introduces algorithms for efficient large-scale network training.

Contribution

It is the first to prove a theorem linking ELM networks with subnetworks and to develop algorithms for parallel and incremental training.

Findings

Theorem demonstrating recursive optimal solutions for ELM subnetworks

Algorithms for parallel and incremental ELM training

Experimental validation of the algorithms' effectiveness

Abstract

A biological neural network is constituted by numerous subnetworks and modules with different functionalities. For an artificial neural network, the relationship between a network and its subnetworks is also important and useful for both theoretical and algorithmic research, i.e. it can be exploited to develop incremental network training algorithm or parallel network training algorithm. In this paper we explore the relationship between an ELM neural network and its subnetworks. To the best of our knowledge, we are the first to prove a theorem that shows an ELM neural network can be scattered into subnetworks and its optimal solution can be constructed recursively by the optimal solutions of these subnetworks. Based on the theorem we also present two algorithms to train a large ELM neural network efficiently: one is a parallel network training algorithm and the other is an incremental network training algorithm. The experimental results demonstrate the usefulness of the theorem and the validity of the developed algorithms.