Eliminating Multicollinearity Issues in Neural Network Ensembles: Incremental, Negatively Correlated, Optimal Convex Blending

TL;DR

This paper presents an incremental ensemble method for neural networks that effectively eliminates multicollinearity issues by optimally blending models with negative correlations, improving accuracy and robustness.

Contribution

The authors introduce a novel incremental algorithm that constructs neural network ensembles with optimal convex blending to prevent multicollinearity.

Findings

Eliminates multicollinearity in neural network ensembles.

Achieves more accurate and robust predictions.

Uses negative correlation blending to enhance ensemble diversity.

Abstract

Given a {features, target} dataset, we introduce an incremental algorithm that constructs an aggregate regressor, using an ensemble of neural networks. It is well known that ensemble methods suffer from the multicollinearity issue, which is the manifestation of redundancy arising mainly due to the common training-dataset. In the present incremental approach, at each stage we optimally blend the aggregate regressor with a newly trained neural network under a convexity constraint which, if necessary, induces negative correlations. Under this framework, collinearity issues do not arise at all, rendering so the method both accurate and robust.