Probabilistically robust conformal prediction

TL;DR

This paper introduces probabilistically robust conformal prediction (PRCP), a method that enhances the reliability of uncertainty quantification in classifiers by accounting for natural and adversarial perturbations, balancing accuracy and robustness.

Contribution

The paper proposes a novel adaptive PRCP algorithm with a dual-threshold approach, extending conformal prediction to be robust against perturbations with theoretical guarantees.

Findings

aPRCP outperforms existing CP methods in robustness and accuracy trade-offs

Experimental results on CIFAR-10, CIFAR-100, and ImageNet validate the effectiveness of aPRCP

Theoretical analysis confirms robust coverage guarantees of the proposed method

Abstract

Conformal prediction (CP) is a framework to quantify uncertainty of machine learning classifiers including deep neural networks. Given a testing example and a trained classifier, CP produces a prediction set of candidate labels with a user-specified coverage (i.e., true class label is contained with high probability). Almost all the existing work on CP assumes clean testing data and there is not much known about the robustness of CP algorithms w.r.t natural/adversarial perturbations to testing examples. This paper studies the problem of probabilistically robust conformal prediction (PRCP) which ensures robustness to most perturbations around clean input examples. PRCP generalizes the standard CP (cannot handle perturbations) and adversarially robust CP (ensures robustness w.r.t worst-case perturbations) to achieve better trade-offs between nominal performance and robustness. We propose a novel adaptive PRCP (aPRCP) algorithm to achieve probabilistically robust coverage. The key idea behind aPRCP is to determine two parallel thresholds, one for data samples and another one for the perturbations on data (aka "quantile-of-quantile" design). We provide theoretical analysis to show that aPRCP algorithm achieves robust coverage. Our experiments on CIFAR-10, CIFAR-100, and ImageNet datasets using deep neural networks demonstrate that aPRCP achieves better trade-offs than state-of-the-art CP and adversarially robust CP algorithms.