Score-oriented loss (SOL) functions

TL;DR

This paper introduces a new class of loss functions for binary classification that are designed to maximize skill scores by leveraging probabilistic confusion matrices, validated through experimental forecasting tasks.

Contribution

It proposes score-oriented loss functions based on probabilistic confusion matrices, enabling automatic skill score maximization during training.

Findings

Loss functions based on probabilistic confusion matrices improve score maximization.

Performance validation on forecasting problems shows significant impact of probability distribution functions.

The approach enhances supervised learning by integrating score optimization into the training process.

Abstract

Loss functions engineering and the assessment of forecasting performances are two crucial and intertwined aspects of supervised machine learning. This paper focuses on binary classification to introduce a class of loss functions that are defined on probabilistic confusion matrices and that allow an automatic and a priori maximization of the skill scores. The performances of these loss functions are validated during the training phase of two experimental forecasting problems, thus showing that the probability distribution function associated with the confusion matrices significantly impacts the outcome of the score maximization process.