Distilling Calibration via Conformalized Credal Inference

TL;DR

This paper presents CD-CI, a low-complexity method for improving uncertainty calibration in edge AI by distilling information from complex models into credal sets, enhancing reliability without high computational costs.

Contribution

The paper introduces a novel calibration distillation technique using credal sets derived from divergence thresholds, suitable for resource-constrained edge AI devices.

Findings

Significantly improves calibration over low-complexity Bayesian methods

Effective on visual and language tasks

Maintains computational efficiency for edge deployment

Abstract

Deploying artificial intelligence (AI) models on edge devices involves a delicate balance between meeting stringent complexity constraints, such as limited memory and energy resources, and ensuring reliable performance in sensitive decision-making tasks. One way to enhance reliability is through uncertainty quantification via Bayesian inference. This approach, however, typically necessitates maintaining and running multiple models in an ensemble, which may exceed the computational limits of edge devices. This paper introduces a low-complexity methodology to address this challenge by distilling calibration information from a more complex model. In an offline phase, predictive probabilities generated by a high-complexity cloud-based model are leveraged to determine a threshold based on the typical divergence between the cloud and edge models. At run time, this threshold is used to construct credal sets -- ranges of predictive probabilities that are guaranteed, with a user-selected confidence level, to include the predictions of the cloud model. The credal sets are obtained through thresholding of a divergence measure in the simplex of predictive probabilities. Experiments on visual and language tasks demonstrate that the proposed approach, termed Conformalized Distillation for Credal Inference (CD-CI), significantly improves calibration performance compared to low-complexity Bayesian methods, such as Laplace approximation, making it a practical and efficient solution for edge AI deployments.