Ranking hierarchical multi-label classification results with mLPRs

TL;DR

This paper introduces a novel ranking method using multidimensional Local Precision Rates (mLPRs) for hierarchical multi-label classification, improving decision accuracy by effectively managing hierarchical constraints and classifier score differences.

Contribution

It proposes a new objective function CATCH and a ranking algorithm HierRank that optimize hierarchical classification performance using mLPRs, addressing a less studied second-stage problem.

Findings

Outperforms state-of-the-art methods on synthetic and real datasets.

Effectively manages hierarchical constraints in multi-label classification.

Improves decision accuracy through mLPR-based ranking.

Abstract

Hierarchical multi-label classification (HMC) has gained considerable attention in recent decades. A seminal line of HMC research addresses the problem in two stages: first, training individual classifiers for each class, then integrating these classifiers to provide a unified set of classification results across classes while respecting the given hierarchy. In this article, we focus on the less attended second-stage question while adhering to the given class hierarchy. This involves addressing a key challenge: how to manage the hierarchical constraint and account for statistical differences in the first-stage classifier scores across different classes to make classification decisions that are optimal under a justifiable criterion. To address this challenge, we introduce a new objective function, called CATCH, to ensure reasonable classification performance. To optimize this function, we propose a decision strategy built on a novel metric, the multidimensional Local Precision Rate (mLPR), which reflects the membership chance of an object in a class given all classifier scores and the class hierarchy. Particularly, we demonstrate that, under certain conditions, transforming the classifier scores into mLPRs and comparing mLPR values for all objects against all classes can, in theory, ensure the class hierarchy and maximize CATCH. In practice, we propose an algorithm HierRank to rank estimated mLPRs under the hierarchical constraint, leading to a ranking that maximizes an empirical version of CATCH. Our approach was evaluated on a synthetic dataset and two real datasets, exhibiting superior performance compared to several state-of-the-art methods in terms of improved decision accuracy.