Calibrating AI Models for Wireless Communications via Conformal Prediction

TL;DR

This paper explores the use of conformal prediction to calibrate AI models in wireless communications, providing formal guarantees of decision reliability across various tasks like demodulation and channel prediction.

Contribution

It introduces the novel application of conformal prediction to communication system AI models, ensuring formal calibration guarantees regardless of underlying data distribution.

Findings

Conformal prediction provides reliable uncertainty quantification in communication AI models.

The method guarantees correct answer inclusion with user-defined probability.

Application to communication tasks improves model trustworthiness.

Abstract

When used in complex engineered systems, such as communication networks, artificial intelligence (AI) models should be not only as accurate as possible, but also well calibrated. A well-calibrated AI model is one that can reliably quantify the uncertainty of its decisions, assigning high confidence levels to decisions that are likely to be correct and low confidence levels to decisions that are likely to be erroneous. This paper investigates the application of conformal prediction as a general framework to obtain AI models that produce decisions with formal calibration guarantees. Conformal prediction transforms probabilistic predictors into set predictors that are guaranteed to contain the correct answer with a probability chosen by the designer. Such formal calibration guarantees hold irrespective of the true, unknown, distribution underlying the generation of the variables of interest, and can be defined in terms of ensemble or time-averaged probabilities. In this paper, conformal prediction is applied for the first time to the design of AI for communication systems in conjunction to both frequentist and Bayesian learning, focusing on demodulation, modulation classification, and channel prediction.