Per-sample Prediction Intervals for Extreme Learning Machines

TL;DR

This paper introduces a fast, robust method for estimating input-dependent prediction intervals in supervised machine learning using Extreme Learning Machines and a variance correction based on a weighted Jackknife approach.

Contribution

It proposes a novel approach to compute input-dependent prediction intervals with Extreme Learning Machines, effectively handling heteroscedastic outputs and large or limited datasets.

Findings

Method is fast and robust to heteroscedasticity

Effective on large datasets and small training sets

Provides reliable input-dependent prediction intervals

Abstract

Prediction intervals in supervised Machine Learning bound the region where the true outputs of new samples may fall. They are necessary in the task of separating reliable predictions of a trained model from near random guesses, minimizing the rate of False Positives, and other problem-specific tasks in applied Machine Learning. Many real problems have heteroscedastic stochastic outputs, which explains the need of input-dependent prediction intervals. This paper proposes to estimate the input-dependent prediction intervals by a separate Extreme Learning Machine model, using variance of its predictions as a correction term accounting for the model uncertainty. The variance is estimated from the model's linear output layer with a weighted Jackknife method. The methodology is very fast, robust to heteroscedastic outputs, and handles both extremely large datasets and insufficient amount of training data.