Exact characterization of {\epsilon}-Safe Decision Regions for exponential family distributions and Multi Cost SVM approximation

TL;DR

This paper defines {}-Safe Decision Regions for reliable classification, proves their analytical form for exponential family distributions, and introduces Multi Cost SVM to approximate these regions in more general, unbalanced data scenarios.

Contribution

It introduces a formal definition of {}-Safe Decision Regions, derives their analytical form for exponential family distributions, and develops Multi Cost SVM to approximate safe regions for broader data types.

Findings

Analytical form of {}-Safe Decision Regions for exponential family distributions.

Multi Cost SVM effectively approximates safe regions in unbalanced datasets.

Experimental results demonstrate the approach's reliability and applicability.

Abstract

Probabilistic guarantees on the prediction of data-driven classifiers are necessary to define models that can be considered reliable. This is a key requirement for modern machine learning in which the goodness of a system is measured in terms of trustworthiness, clearly dividing what is safe from what is unsafe. The spirit of this paper is exactly in this direction. First, we introduce a formal definition of {\epsilon}-Safe Decision Region, a subset of the input space in which the prediction of a target (safe) class is probabilistically guaranteed. Second, we prove that, when data come from exponential family distributions, the form of such a region is analytically determined and controllable by design parameters, i.e. the probability of sampling the target class and the confidence on the prediction. However, the request of having exponential data is not always possible. Inspired by this limitation, we developed Multi Cost SVM, an SVM based algorithm that approximates the safe region and is also able to handle unbalanced data. The research is complemented by experiments and code available for reproducibility.