Scalable computation of prediction intervals for neural networks via matrix sketching

TL;DR

This paper introduces a scalable method for estimating prediction intervals in neural networks using matrix sketching, which is efficient and applicable to pre-trained models, outperforming some existing approaches.

Contribution

The authors propose a novel, computationally efficient algorithm based on the delta method and matrix sketching to produce approximate prediction intervals for neural networks.

Findings

Competitive with state-of-the-art methods on UCI datasets

Does not require modifying the trained neural network

Achieves efficiency through matrix sketching

Abstract

Accounting for the uncertainty in the predictions of modern neural networks is a challenging and important task in many domains. Existing algorithms for uncertainty estimation require modifying the model architecture and training procedure (e.g., Bayesian neural networks) or dramatically increase the computational cost of predictions such as approaches based on ensembling. This work proposes a new algorithm that can be applied to a given trained neural network and produces approximate prediction intervals. The method is based on the classical delta method in statistics but achieves computational efficiency by using matrix sketching to approximate the Jacobian matrix. The resulting algorithm is competitive with state-of-the-art approaches for constructing predictive intervals on various regression datasets from the UCI repository.