Performance Metrics (Error Measures) in Machine Learning Regression, Forecasting and Prognostics: Properties and Typology

TL;DR

This paper provides a comprehensive overview and classification of performance metrics used in machine learning regression, forecasting, and prognostics, proposing a new typology based on their structural properties.

Contribution

It introduces a novel typology of performance metrics, categorizing them into primary, extended, composite, and hybrid sets to enhance understanding and selection.

Findings

Proposed a framework with four metric categories.

Identified three key components influencing metric structure.

Facilitates better metric selection in regression and forecasting.

Abstract

Performance metrics (error measures) are vital components of the evaluation frameworks in various fields. The intention of this study was to overview of a variety of performance metrics and approaches to their classification. The main goal of the study was to develop a typology that will help to improve our knowledge and understanding of metrics and facilitate their selection in machine learning regression, forecasting and prognostics. Based on the analysis of the structure of numerous performance metrics, we propose a framework of metrics which includes four (4) categories: primary metrics, extended metrics, composite metrics, and hybrid sets of metrics. The paper identified three (3) key components (dimensions) that determine the structure and properties of primary metrics: method of determining point distance, method of normalization, method of aggregation of point distances over a data set.