Selecting a classification performance measure: matching the measure to the problem

TL;DR

Choosing the right performance measure for classification tasks is crucial, as it should align with the specific goals of the application to accurately evaluate and compare different methods.

Contribution

This paper emphasizes the importance of matching classification performance measures to the specific aims of the research or application.

Findings

Different performance measures suit different classification goals

Matching measure properties to application aims improves evaluation accuracy

Highlights the need for careful measure selection in classification tasks

Abstract

The problem of identifying to which of a given set of classes objects belong is ubiquitous, occurring in many research domains and application areas, including medical diagnosis, financial decision making, online commerce, and national security. But such assignments are rarely completely perfect, and classification errors occur. This means it is necessary to compare classification methods and algorithms to decide which is ``best'' for any particular problem. However, just as there are many different classification methods, so there are many different ways of measuring their performance. It is thus vital to choose a measure of performance which matches the aims of the research or application. This paper is a contribution to the growing literature on the relative merits of different performance measures. Its particular focus is the critical importance of matching the properties of the measure to the aims for which the classification is being made.