Adaptive Neural Network Ensemble Using Frequency Distribution

TL;DR

This paper introduces an adaptive neural network ensemble method using frequency distribution analysis to identify core predictions, enhancing accuracy especially in nonlinear problems with limited data, and proposes an adaptive sampling strategy for further improvement.

Contribution

The study presents a novel frequency distribution-based ensemble method and an adaptive sampling strategy to improve neural network ensemble accuracy under data scarcity and nonlinearity.

Findings

Outperforms Kriging and existing ensemble methods in prediction accuracy.

Adaptive sampling strategy enhances ensemble performance effectively.

Frequency distribution approach reduces prediction variance and uncertainty.

Abstract

Neural network (NN) ensembles can reduce large prediction variance of NN and improve prediction accuracy. For highly nonlinear problems with insufficient data set, the prediction accuracy of NN models becomes unstable, resulting in a decrease in the accuracy of ensembles. Therefore, this study proposes a frequency distribution-based ensemble that identifies core prediction values, which are expected to be concentrated near the true prediction value. The frequency distribution-based ensemble classifies core prediction values supported by multiple prediction values by conducting statistical analysis with a frequency distribution, which is based on various prediction values obtained from a given prediction point. The frequency distribution-based ensemble can improve predictive performance by excluding prediction values with low accuracy and coping with the uncertainty of the most frequent value. An adaptive sampling strategy that sequentially adds samples based on the core prediction variance calculated as the variance of the core prediction values is proposed to improve the predictive performance of the frequency distribution-based ensemble efficiently. Results of various case studies show that the prediction accuracy of the frequency distribution-based ensemble is higher than that of Kriging and other existing ensemble methods. In addition, the proposed adaptive sampling strategy effectively improves the predictive performance of the frequency distribution-based ensemble compared with the previously developed space-filling and prediction variance-based strategies.