Bottom-up Hierarchical Classification Using Confusion-based Logit Compression

TL;DR

This paper introduces a robust logit compression method for hierarchical classification that improves label posterior estimation when validation data is scarce, enhancing performance without overfitting.

Contribution

It presents a novel logit vector compression technique based on label confusions, enabling effective hierarchical classification with limited validation data.

Findings

Outperforms existing compression methods across various validation set sizes.

Maintains strong hierarchical classification accuracy with minimal validation data.

Reduces overfitting risk by using validation data solely for posterior estimation.

Abstract

In this work, we propose a method to efficiently compute label posteriors of a base flat classifier in the presence of few validation examples within a bottom-up hierarchical inference framework. A stand-alone validation set (not used to train the base classifier) is preferred for posterior estimation to avoid overfitting the base classifier, however a small validation set limits the number of features one can effectively use. We propose a simple, yet robust, logit vector compression approach based on generalized logits and label confusions for the task of label posterior estimation within the context of hierarchical classification. Extensive comparative experiments with other compression techniques are provided across multiple sized validation sets, and a comparison with related hierarchical classification approaches is also conducted. The proposed approach mitigates the problem of not having enough validation examples for reliable posterior estimation while maintaining strong hierarchical classification performance.