Onboard Out-of-Calibration Detection of Deep Learning Models using Conformal Prediction

TL;DR

This paper introduces a method using conformal prediction to detect when deep learning models are out-of-calibration, especially in remote sensing applications, by relating model uncertainty to prediction set size.

Contribution

It demonstrates how conformal prediction can be used to identify out-of-calibration in deep models through uncertainty and prediction set analysis.

Findings

Conformal prediction relates to model uncertainty in deep learning.

Out-of-calibration detection is effective using prediction set size.

Method applied to popular models on remote sensing datasets.

Abstract

The black box nature of deep learning models complicate their usage in critical applications such as remote sensing. Conformal prediction is a method to ensure trust in such scenarios. Subject to data exchangeability, conformal prediction provides finite sample coverage guarantees in the form of a prediction set that is guaranteed to contain the true class within a user defined error rate. In this letter we show that conformal prediction algorithms are related to the uncertainty of the deep learning model and that this relation can be used to detect if the deep learning model is out-of-calibration. Popular classification models like Resnet50, Densenet161, InceptionV3, and MobileNetV2 are applied on remote sensing datasets such as the EuroSAT to demonstrate how under noisy scenarios the model outputs become untrustworthy. Furthermore an out-of-calibration detection procedure relating the model uncertainty and the average size of the conformal prediction set is presented.