Reliable Evaluation of Neural Network for Multiclass Classification of Real-world Data

TL;DR

This paper systematically evaluates neural network performance for multiclass classification on real-world, skewed datasets, revealing limitations of traditional accuracy metrics and exploring multiple evaluation parameters.

Contribution

It introduces a comprehensive evaluation framework using twelve parameters to assess neural network performance on diverse real-world datasets with class imbalance.

Findings

Accuracy alone may be unreliable for skewed data

Multiple evaluation parameters provide better performance insights

Neural networks' effectiveness varies across different dataset types

Abstract

This paper presents a systematic evaluation of Neural Network (NN) for classification of real-world data. In the field of machine learning, it is often seen that a single parameter that is 'predictive accuracy' is being used for evaluating the performance of a classifier model. However, this parameter might not be considered reliable given a dataset with very high level of skewness. To demonstrate such behavior, seven different types of datasets have been used to evaluate a Multilayer Perceptron (MLP) using twelve(12) different parameters which include micro- and macro-level estimation. In the present study, the most common problem of prediction called 'multiclass' classification has been considered. The results that are obtained for different parameters for each of the dataset could demonstrate interesting findings to support the usability of these set of performance evaluation parameters.