Adaptive Online Sequential ELM for Concept Drift Tackling

TL;DR

This paper introduces AOS-ELM, an adaptive online learning method that effectively handles various types of concept drift in classification and regression tasks, demonstrated on multiple datasets.

Contribution

It presents a simple, unified adaptive scheme for OS-ELM that improves handling of concept drift without increasing hidden nodes.

Findings

Higher kappa value than multiclassifier ELM ensemble.

Effective in real, virtual, and hybrid drift scenarios.

Addresses hidden node issues using pseudoinverse rank indicator.

Abstract

A machine learning method needs to adapt to over time changes in the environment. Such changes are known as concept drift. In this paper, we propose concept drift tackling method as an enhancement of Online Sequential Extreme Learning Machine (OS-ELM) and Constructive Enhancement OS-ELM (CEOS-ELM) by adding adaptive capability for classification and regression problem. The scheme is named as adaptive OS-ELM (AOS-ELM). It is a single classifier scheme that works well to handle real drift, virtual drift, and hybrid drift. The AOS-ELM also works well for sudden drift and recurrent context change type. The scheme is a simple unified method implemented in simple lines of code. We evaluated AOS-ELM on regression and classification problem by using concept drift public data set (SEA and STAGGER) and other public data sets such as MNIST, USPS, and IDS. Experiments show that our method gives higher kappa value compared to the multiclassifier ELM ensemble. Even though AOS-ELM in practice does not need hidden nodes increase, we address some issues related to the increasing of the hidden nodes such as error condition and rank values. We propose taking the rank of the pseudoinverse matrix as an indicator parameter to detect underfitting condition.