A unifying view for performance measures in multi-class prediction

TL;DR

This paper explores the relationships among various multi-class performance measures, especially linking Confusion Entropy with Matthews Correlation Coefficient, supported by computational and theoretical analysis.

Contribution

It establishes a strong monotone relation between Confusion Entropy and Matthews Correlation Coefficient, unifying different performance metrics in multi-class classification.

Findings

Confusion Entropy is strongly related to Matthews Correlation Coefficient.

The relation between these measures is monotone and supported by computational evidence.

A theoretical explanation for this relation is provided.

Abstract

In the last few years, many different performance measures have been introduced to overcome the weakness of the most natural metric, the Accuracy. Among them, Matthews Correlation Coefficient has recently gained popularity among researchers not only in machine learning but also in several application fields such as bioinformatics. Nonetheless, further novel functions are being proposed in literature. We show that Confusion Entropy, a recently introduced classifier performance measure for multi-class problems, has a strong (monotone) relation with the multi-class generalization of a classical metric, the Matthews Correlation Coefficient. Computational evidence in support of the claim is provided, together with an outline of the theoretical explanation.