Compact Learning for Multi-Label Classification

TL;DR

This paper introduces a novel compact learning framework for multi-label classification that jointly embeds features and labels to improve dependency modeling and label space recovery, addressing limitations of existing label compression methods.

Contribution

It proposes a versatile compact learning framework with a specific implementation called CMLL that enhances multi-label classification by mutual feature-label embedding.

Findings

CMLL outperforms existing methods in experiments.

The framework effectively captures label dependencies.

Theoretical analysis supports various embedding techniques.

Abstract

Multi-label classification (MLC) studies the problem where each instance is associated with multiple relevant labels, which leads to the exponential growth of output space. MLC encourages a popular framework named label compression (LC) for capturing label dependency with dimension reduction. Nevertheless, most existing LC methods failed to consider the influence of the feature space or misguided by original problematic features, so that may result in performance degeneration. In this paper, we present a compact learning (CL) framework to embed the features and labels simultaneously and with mutual guidance. The proposal is a versatile concept, hence the embedding way is arbitrary and independent of the subsequent learning process. Following its spirit, a simple yet effective implementation called compact multi-label learning (CMLL) is proposed to learn a compact low-dimensional representation for both spaces. CMLL maximizes the dependence between the embedded spaces of the labels and features, and minimizes the loss of label space recovery concurrently. Theoretically, we provide a general analysis for different embedding methods. Practically, we conduct extensive experiments to validate the effectiveness of the proposed method.