The MBPEP: a deep ensemble pruning algorithm providing high quality uncertainty prediction

TL;DR

The paper introduces MBPEP, a deep ensemble pruning method that enhances uncertainty prediction accuracy by optimizing ensemble size and using novel loss functions, applicable to classification and regression tasks.

Contribution

It proposes a margin-based Pareto pruning algorithm with new loss functions to improve uncertainty estimation in deep ensemble models.

Findings

MBPEP achieves small prediction interval widths and low errors.

It performs well on datasets with unknown distributions.

Enhances multi-task learning performance.

Abstract

Machine learning algorithms have been effectively applied into various real world tasks. However, it is difficult to provide high-quality machine learning solutions to accommodate an unknown distribution of input datasets; this difficulty is called the uncertainty prediction problems. In this paper, a margin-based Pareto deep ensemble pruning (MBPEP) model is proposed. It achieves the high-quality uncertainty estimation with a small value of the prediction interval width (MPIW) and a high confidence of prediction interval coverage probability (PICP) by using deep ensemble networks. In addition to these networks, unique loss functions are proposed, and these functions make the sub-learners available for standard gradient descent learning. Furthermore, the margin criterion fine-tuning-based Pareto pruning method is introduced to optimize the ensembles. Several experiments including predicting uncertainties of classification and regression are conducted to analyze the performance of MBPEP. The experimental results show that MBPEP achieves a small interval width and a low learning error with an optimal number of ensembles. For the real-world problems, MBPEP performs well on input datasets with unknown distributions datasets incomings and improves learning performance on a multi task problem when compared to that of each single model.