# Confidence intervals for class prevalences under prior probability shift

## TL;DR

This paper investigates the construction of confidence and prediction intervals for class prevalence estimates under prior probability shift, examining the importance of classifier discriminatory power and the distinction between interval types through simulation.

## Contribution

It provides new insights into the necessity of differentiating confidence and prediction intervals and the impact of classifier power on prevalence estimation accuracy under dataset shift.

## Key findings

- Confidence and prediction intervals may not need to be distinguished in practice.
- Classifier discriminatory power influences the accuracy of prevalence estimates.
- Simulation results clarify conditions affecting interval construction and estimator performance.

## Abstract

Point estimation of class prevalences in the presence of data set shift has been a popular research topic for more than two decades. Less attention has been paid to the construction of confidence and prediction intervals for estimates of class prevalences. One little considered question is whether or not it is necessary for practical purposes to distinguish confidence and prediction intervals. Another question so far not yet conclusively answered is whether or not the discriminatory power of the classifier or score at the basis of an estimation method matters for the accuracy of the estimates of the class prevalences. This paper presents a simulation study aimed at shedding some light on these and other related questions.