To Each Metric Its Decoding: Post-Hoc Optimal Decision Rules of Probabilistic Hierarchical Classifiers

TL;DR

This paper develops a framework for optimal decision rules in hierarchical classification, aligning decoding strategies with specific evaluation metrics to improve classifier performance in complex, real-world tasks.

Contribution

It introduces a universal approach for optimal decoding in hierarchical classifiers, especially for hierarchical $hF_{eta}$ scores, with algorithms applicable to various prediction complexities.

Findings

Optimal decision rules outperform heuristic methods.

Strategies are effective in underdetermined prediction scenarios.

Empirical results demonstrate improved classifier reliability.

Abstract

Hierarchical classification offers an approach to incorporate the concept of mistake severity by leveraging a structured, labeled hierarchy. However, decoding in such settings frequently relies on heuristic decision rules, which may not align with task-specific evaluation metrics. In this work, we propose a framework for the optimal decoding of an output probability distribution with respect to a target metric. We derive optimal decision rules for increasingly complex prediction settings, providing universal algorithms when candidates are limited to the set of nodes. In the most general case of predicting a subset of nodes, we focus on rules dedicated to the hierarchical $hF_{\beta}$ scores, tailored to hierarchical settings. To demonstrate the practical utility of our approach, we conduct extensive empirical evaluations, showcasing the superiority of our proposed optimal strategies, particularly in underdetermined scenarios. These results highlight the potential of our methods to enhance the performance and reliability of hierarchical classifiers in real-world applications. The code is available at https://github.com/RomanPlaud/hierarchical_decision_rules