Neurosymbolic Conformal Classification

TL;DR

This paper explores the integration of neurosymbolic AI and conformal prediction to enhance trustworthiness in machine learning by providing statistical guarantees and combining reasoning with neural learning.

Contribution

It introduces neurosymbolic conformal prediction techniques, demonstrating how these methods can complement each other to improve confidence guarantees in AI systems.

Findings

Multiple neurosymbolic conformal prediction methods are proposed.

The techniques vary in confidence set size and computational complexity.

The approaches are distribution-free and model-agnostic.

Abstract

The last decades have seen a drastic improvement of Machine Learning (ML), mainly driven by Deep Learning (DL). However, despite the resounding successes of ML in many domains, the impossibility to provide guarantees of conformity and the fragility of ML systems (faced with distribution shifts, adversarial attacks, etc.) have prevented the design of trustworthy AI systems. Several research paths have been investigated to mitigate this fragility and provide some guarantees regarding the behavior of ML systems, among which are neurosymbolic AI and conformal prediction. Neurosymbolic artificial intelligence is a growing field of research aiming to combine neural network learning capabilities with the reasoning abilities of symbolic systems. One of the objective of this hybridization can be to provide theoritical guarantees that the output of the system will comply with some prior knowledge. Conformal prediction is a set of techniques that enable to take into account the uncertainty of ML systems by transforming the unique prediction into a set of predictions, called a confidence set. Interestingly, this comes with statistical guarantees regarding the presence of the true label inside the confidence set. Both approaches are distribution-free and model-agnostic. In this paper, we see how these two approaches can complement one another. We introduce several neurosymbolic conformal prediction techniques and explore their different characteristics (size of confidence sets, computational complexity, etc.).