Robust Stochastic Configuration Networks with Kernel Density Estimation

TL;DR

This paper introduces robust stochastic configuration networks (RSCNs) that incorporate kernel density estimation to effectively handle noisy data and outliers, improving model robustness and generalization in regression tasks.

Contribution

The paper develops RSCNs that integrate KDE-based penalty weights and an alternating optimization process, offering a novel approach to enhance robustness against uncertain data in neural network models.

Findings

RSCNs outperform existing robust neural models in various tests.

The method effectively reduces the impact of noisy samples and outliers.

Demonstrates strong potential for real-world data regression applications.

Abstract

Neural networks have been widely used as predictive models to fit data distribution, and they could be implemented through learning a collection of samples. In many applications, however, the given dataset may contain noisy samples or outliers which may result in a poor learner model in terms of generalization. This paper contributes to a development of robust stochastic configuration networks (RSCNs) for resolving uncertain data regression problems. RSCNs are built on original stochastic configuration networks with weighted least squares method for evaluating the output weights, and the input weights and biases are incrementally and randomly generated by satisfying with a set of inequality constrains. The kernel density estimation (KDE) method is employed to set the penalty weights for each training samples, so that some negative impacts, caused by noisy data or outliers, on the resulting learner model can be reduced. The alternating optimization technique is applied for updating a RSCN model with improved penalty weights computed from the kernel density estimation function. Performance evaluation is carried out by a function approximation, four benchmark datasets and a case study on engineering application. Comparisons to other robust randomised neural modelling techniques, including the probabilistic robust learning algorithm for neural networks with random weights and improved RVFL networks, indicate that the proposed RSCNs with KDE perform favourably and demonstrate good potential for real-world applications.