An improved hybrid regularization approach for extreme learning machine

TL;DR

This paper introduces a novel hybrid regularization method for extreme learning machines (ELM) that enhances classification accuracy, sparsity, and stability by combining $ ext{L}_2$ and $ ext{L}_{0.5}$ regularizations, solved via an iterative fixed point algorithm.

Contribution

The paper proposes a new $ ext{L}_2$-$ ext{L}_{0.5}$ regularization model for ELM and develops an iterative algorithm to optimize it, improving performance over existing models.

Findings

The $ ext{L}_2$-$ ext{L}_{0.5}$-ELM outperforms other models in accuracy.

The method achieves higher sparsity and stability.

Convergence of the algorithm is theoretically analyzed.

Abstract

Extreme learning machine (ELM) is a network model that arbitrarily initializes the first hidden layer and can be computed speedily. In order to improve the classification performance of ELM, a $\ell_2$ and $\ell_{0.5}$ regularization ELM model ($\ell_{2}$-$\ell_{0.5}$-ELM) is proposed in this paper. An iterative optimization algorithm of the fixed point contraction mapping is applied to solve the $\ell_{2}$-$\ell_{0.5}$-ELM model. The convergence and sparsity of the proposed method are discussed and analyzed under reasonable assumptions. The performance of the proposed $\ell_{2}$-$\ell_{0.5}$-ELM method is compared with BP, SVM, ELM, $\ell_{0.5}$-ELM, $\ell_{1}$-ELM, $\ell_{2}$-ELM and $\ell_{2}$-$\ell_{1}$ELM, the results show that the prediction accuracy, sparsity, and stability of the $\ell_{2}$-$\ell_{0.5}$-ELM are better than the other $7$ models.