Derivations of Normalized Mutual Information in Binary Classifications

TL;DR

This paper introduces normalized mutual information as a new, well-defined metric for evaluating binary classifiers, deriving its relationships with traditional performance measures like accuracy, precision, and recall.

Contribution

It provides closed-form relations between normalized mutual information and common classifier performance metrics, revealing its nonlinear dependence and potential for improved evaluation.

Findings

Normalized mutual information is a compact, well-defined evaluation metric.

Derived explicit relations between NI and accuracy, precision, recall.

Revealed NI as a set of nonlinear functions of traditional metrics.

Abstract

This correspondence studies the basic problem of classifications - how to evaluate different classifiers. Although the conventional performance indexes, such as accuracy, are commonly used in classifier selection or evaluation, information-based criteria, such as mutual information, are becoming popular in feature/model selections. In this work, we propose to assess classifiers in terms of normalized mutual information (NI), which is novel and well defined in a compact range for classifier evaluation. We derive close-form relations of normalized mutual information with respect to accuracy, precision, and recall in binary classifications. By exploring the relations among them, we reveal that NI is actually a set of nonlinear functions, with a concordant power-exponent form, to each performance index. The relations can also be expressed with respect to precision and recall, or to false alarm and hitting rate (recall).