Discriminative Learning of Prediction Intervals

TL;DR

This paper introduces a discriminative learning framework for constructing prediction intervals that optimizes expected error rates under size constraints, offering improved accuracy and efficiency over traditional methods.

Contribution

It proposes a novel discriminative approach with PAC-style guarantees for prediction intervals, addressing computational challenges with a convex surrogate.

Findings

The method achieves lower average interval sizes.

It provides finite-sample guarantees.

Experimental results validate improved accuracy.

Abstract

In this work we consider the task of constructing prediction intervals in an inductive batch setting. We present a discriminative learning framework which optimizes the expected error rate under a budget constraint on the interval sizes. Most current methods for constructing prediction intervals offer guarantees for a single new test point. Applying these methods to multiple test points can result in a high computational overhead and degraded statistical guarantees. By focusing on expected errors, our method allows for variability in the per-example conditional error rates. As we demonstrate both analytically and empirically, this flexibility can increase the overall accuracy, or alternatively, reduce the average interval size. While the problem we consider is of a regressive flavor, the loss we use is combinatorial. This allows us to provide PAC-style, finite-sample guarantees. Computationally, we show that our original objective is NP-hard, and suggest a tractable convex surrogate. We conclude with a series of experimental evaluations.