Broad stochastic configuration residual learning system for norm-convergent universal approximation

TL;DR

This paper introduces BSCRLS, a new neural network learning system that guarantees norm convergence for universal approximation, addressing limitations of previous randomized models and demonstrating superior performance in dust detection tasks.

Contribution

The paper proposes BSCRLS, a stochastic residual learning system with a supervisory mechanism ensuring norm convergence, improving the reliability of universal approximation in randomized neural networks.

Findings

BSCRLS achieves universal approximation with norm convergence.

Experimental results show BSCRLS outperforms 13 existing algorithms.

BSCRLS is effective in dust detection on real datasets.

Abstract

Universal approximation serves as the foundation of neural network learning algorithms. However, some networks establish their universal approximation property by demonstrating that the iterative errors converge in probability measure rather than the more rigorous norm convergence, which makes the universal approximation property of randomized learning networks highly sensitive to random parameter selection, Broad residual learning system (BRLS), as a member of randomized learning models, also encounters this issue. We theoretically demonstrate the limitation of its universal approximation property, that is, the iterative errors do not satisfy norm convergence if the selection of random parameters is inappropriate and the convergence rate meets certain conditions. To address this issue, we propose the broad stochastic configuration residual learning system (BSCRLS) algorithm, which features a novel supervisory mechanism adaptively constraining the range settings of random parameters on the basis of BRLS framework, Furthermore, we prove the universal approximation theorem of BSCRLS based on the more stringent norm convergence. Three versions of incremental BSCRLS algorithms are presented to satisfy the application requirements of various network updates. Solar panels dust detection experiments are performed on publicly available dataset and compared with 13 deep and broad learning algorithms. Experimental results reveal the effectiveness and superiority of BSCRLS algorithms.