Extreme Learning Machine design for dealing with unrepresentative features

TL;DR

This paper explores the benefits of using larger hidden layer sizes in Extreme Learning Machines for unrepresentative features, proposing a pruning method to manage computational costs and demonstrating improved EEG signal classification performance.

Contribution

It introduces a novel approach to select larger hidden layer sizes in ELMs and presents a pruning algorithm to reduce computational burden, improving performance on EEG data.

Findings

Larger hidden layers can improve ELM performance with unrepresentative features.

Pruning reduces computational costs while maintaining accuracy.

Experimental results show enhanced EEG classification accuracy.

Abstract

Extreme Learning Machines (ELMs) have become a popular tool in the field of Artificial Intelligence due to their very high training speed and generalization capabilities. Another advantage is that they have a single hyper-parameter that must be tuned up: the number of hidden nodes. Most traditional approaches dictate that this parameter should be chosen smaller than the number of available training samples in order to avoid over-fitting. In fact, it has been proved that choosing the number of hidden nodes equal to the number of training samples yields a perfect training classification with probability 1 (w.r.t. the random parameter initialization). In this article we argue that in spite of this, in some cases it may be beneficial to choose a much larger number of hidden nodes, depending on certain properties of the data. We explain why this happens and show some examples to illustrate how the model behaves. In addition, we present a pruning algorithm to cope with the additional computational burden associated to the enlarged ELM. Experimental results using electroencephalography (EEG) signals show an improvement in performance with respect to traditional ELM approaches, while diminishing the extra computing time associated to the use of large architectures.