Design-based conformal prediction

TL;DR

This paper introduces conformal prediction methods tailored for complex survey data, providing finite-sample coverage guarantees and demonstrating practical application through simulations and real data examples.

Contribution

It extends conformal prediction to complex survey designs within a design-based inference framework, highlighting new applications for survey methodologists.

Findings

Empirical results confirm finite-sample coverage guarantees.

Simulations demonstrate effectiveness across various survey designs.

Real data example showcases practical applicability.

Abstract

Conformal prediction is an assumption-lean approach to generating distribution-free prediction intervals or sets, for nearly arbitrary predictive models, with guaranteed finite-sample coverage. Conformal methods are an active research topic in statistics and machine learning, but only recently have they been extended to non-exchangeable data. In this paper, we invite survey methodologists to begin using and contributing to conformal methods. We introduce how conformal prediction can be applied to data from several common complex sample survey designs, under a framework of design-based inference for a finite population, and we point out gaps where survey methodologists could fruitfully apply their expertise. Our simulations empirically bear out the theoretical guarantees of finite-sample coverage, and our real-data example demonstrates how conformal prediction can be applied to complex sample survey data in practice.