Predictive Inference Is Free with the Jackknife+-after-Bootstrap
Byol Kim, Chen Xu, Rina Foygel Barber

TL;DR
This paper introduces the jackknife+-after-bootstrap (J+aB), a method for constructing valid predictive intervals from ensemble models that is assumption-free and cost-effective, with proven coverage guarantees.
Contribution
The paper proposes the J+aB procedure, a novel, assumption-lean method for predictive inference using bootstrap ensembles, with theoretical coverage guarantees and no additional model fitting costs.
Findings
J+aB provides valid predictive coverage without distributional assumptions.
Numerical experiments confirm the accuracy and coverage of J+aB intervals.
J+aB is computationally free, relying only on existing bootstrap samples.
Abstract
Ensemble learning is widely used in applications to make predictions in complex decision problems---for example, averaging models fitted to a sequence of samples bootstrapped from the available training data. While such methods offer more accurate, stable, and robust predictions and model estimates, much less is known about how to perform valid, assumption-lean inference on the output of these types of procedures. In this paper, we propose the jackknife+-after-bootstrap (J+aB), a procedure for constructing a predictive interval, which uses only the available bootstrapped samples and their corresponding fitted models, and is therefore "free" in terms of the cost of model fitting. The J+aB offers a predictive coverage guarantee that holds with no assumptions on the distribution of the data, the nature of the fitted model, or the way in which the ensemble of models are aggregated---at…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
Taxonomy
TopicsMachine Learning and Data Classification · Machine Learning in Healthcare · Time Series Analysis and Forecasting
