Comprehensive Comparative Study of Multi-Label Classification Methods

TL;DR

This paper presents a comprehensive empirical comparison of 26 multi-label classification methods across 42 diverse datasets, providing insights into their performance and efficiency using rigorous evaluation standards.

Contribution

It offers the most extensive evaluation to date of MLC methods, covering a wide range of algorithms, datasets, and performance metrics, with a standardized methodology.

Findings

RFPCT, RFDTBR, ECCJ48, EBRJ48, and AdaBoostMH are top performers.

The study highlights the importance of diverse datasets and evaluation metrics.

Method comparisons should consider multiple criteria for robustness.

Abstract

Multi-label classification (MLC) has recently received increasing interest from the machine learning community. Several studies provide reviews of methods and datasets for MLC and a few provide empirical comparisons of MLC methods. However, they are limited in the number of methods and datasets considered. This work provides a comprehensive empirical study of a wide range of MLC methods on a plethora of datasets from various domains. More specifically, our study evaluates 26 methods on 42 benchmark datasets using 20 evaluation measures. The adopted evaluation methodology adheres to the highest literature standards for designing and executing large scale, time-budgeted experimental studies. First, the methods are selected based on their usage by the community, assuring representation of methods across the MLC taxonomy of methods and different base learners. Second, the datasets cover a wide range of complexity and domains of application. The selected evaluation measures assess the predictive performance and the efficiency of the methods. The results of the analysis identify RFPCT, RFDTBR, ECCJ48, EBRJ48 and AdaBoostMH as best performing methods across the spectrum of performance measures. Whenever a new method is introduced, it should be compared to different subsets of MLC methods, determined on the basis of the different evaluation criteria.