A robust regularized extreme learning machine for regression problems based on self-adaptive accelerated extra-gradient algorithm

TL;DR

This paper introduces a new robust regularized extreme learning machine for regression that employs a self-adaptive accelerated extra-gradient algorithm, improving efficiency and convergence over traditional methods.

Contribution

It develops a novel self-adaptive accelerated extra-gradient algorithm for ELM, enhancing computational efficiency and convergence in regression tasks.

Findings

The proposed algorithm converges strongly with a linear rate.

It outperforms existing algorithms in computational experiments.

The method effectively reduces modeling errors and output weight norms.

Abstract

The Extreme Learning Machine (ELM) technique is a machine learning approach for constructing feed-forward neural networks with a single hidden layer and their models. The ELM model can be constructed while being trained by concurrently reducing both the modeling errors and the norm of the output weights. Usually, the squared loss is widely utilized in the objective function of ELM, which can be treated as a LASSO problem and hence applying the fast iterative shrinkage thresholding algorithm (FISTA). However, in this paper, the minimization problem is solved from a variational inequalities perspective giving rise to improved algorithms that are more efficient than the FISTA. A fast general extra-gradient algorithm which is a form of first-order algorithm is developed with inertial acceleration techniques and variable stepsize which is updated at every iteration. The strong convergence and linear rate of convergence of the proposed algorithm are established under mild conditions. In terms of experiments, two experiments were presented to illustrate the computational advantage of the proposed method to other algorithms.