Asymptotic consistency and order specification for logistic classifier chains in multi-label learning

TL;DR

This paper studies the asymptotic properties of logistic classifier chains in multi-label learning, focusing on conditions for convergence, the impact of label order, and methods for optimal ordering to improve model accuracy.

Contribution

It provides the first analysis of asymptotic consistency and order effects in logistic classifier chains, proposing a procedure for optimal label ordering based on model specification measures.

Findings

Convergence of the estimated joint distribution to the true distribution under certain conditions.

The order of labels significantly affects the accuracy of joint probability estimation.

Optimal ordering improves model specification and estimation accuracy.

Abstract

Classifier chains are popular and effective method to tackle a multi-label classification problem. The aim of this paper is to study the asymptotic properties of the chain model in which the conditional probabilities are of the logistic form. In particular we find conditions on the number of labels and the distribution of feature vector under which the estimated mode of the joint distribution of labels converges to the true mode. Best of our knowledge, this important issue has not yet been studied in the context of multi-label learning. We also investigate how the order of model building in a chain influences the estimation of the joint distribution of labels. We establish the link between the problem of incorrect ordering in the chain and incorrect model specification. We propose a procedure of determining the optimal ordering of labels in the chain, which is based on using measures of correct specification and allows to find the ordering such that the consecutive logistic models are best possibly specified. The other important question raised in this paper is how accurately can we estimate the joint posterior probability when the ordering of labels is wrong or the logistic models in the chain are incorrectly specified. The numerical experiments illustrate the theoretical results.