Design-based individual prediction

TL;DR

This paper introduces a design-based approach for individual prediction that leverages expected cross-validation results, ensuring valid inference of prediction errors considering sampling and sample-splitting designs.

Contribution

It develops a novel framework for valid inference of prediction errors in individual prediction, accounting for sampling and cross-validation designs, regardless of model ensemble or weighting.

Findings

Provides a method for valid inference of prediction errors.

Applicable to ensemble and weighted average predictors.

Ensures inference validity under complex sampling designs.

Abstract

A design-based individual prediction approach is developed based on the expected cross-validation results, given the sampling design and the sample-splitting design for cross-validation. Whether the predictor is selected from an ensemble of models or a weighted average of them, valid inference of the unobserved prediction errors is defined and obtained with respect to the sampling design, while outcomes and features are treated as constants.