Absolute Evaluation Measures for Machine Learning: A Survey

TL;DR

This survey reviews absolute evaluation measures across various machine learning tasks, providing guidance on their appropriate use to improve model assessment and comparison.

Contribution

It offers a comprehensive overview of absolute evaluation metrics for classification, clustering, regression, and ranking, organized by learning problem type.

Findings

Classification metrics are well-studied, but less so for other tasks.

Provides guidance on selecting appropriate evaluation measures.

Facilitates better comparison of ML models across domains.

Abstract

Machine Learning is a diverse field applied across various domains such as computer science, social sciences, medicine, chemistry, and finance. This diversity results in varied evaluation approaches, making it difficult to compare models effectively. Absolute evaluation measures offer a practical solution by assessing a model's performance on a fixed scale, independent of reference models and data ranges, enabling explicit comparisons. However, many commonly used measures are not universally applicable, leading to a lack of comprehensive guidance on their appropriate use. This survey addresses this gap by providing an overview of absolute evaluation metrics in ML, organized by the type of learning problem. While classification metrics have been extensively studied, this work also covers clustering, regression, and ranking metrics. By grouping these measures according to the specific ML challenges they address, this survey aims to equip practitioners with the tools necessary to select appropriate metrics for their models. The provided overview thus improves individual model evaluation and facilitates meaningful comparisons across different models and applications.