Multi-label Learning with Random Circular Vectors

TL;DR

This paper introduces a novel approach for extreme multi-label classification using random circular vectors, which improves label encoding and retrieval, leading to better performance and significantly smaller output layers in deep neural networks.

Contribution

It proposes using complex-valued circular vectors for label encoding in DNNs, enhancing efficiency and accuracy in XMC tasks compared to traditional real-valued vectors.

Findings

Circular vectors outperform real-valued vectors in label encoding capacity.

The method achieves significant performance improvements on XMC datasets.

Output layer size is reduced by up to 99% with circular vectors.

Abstract

The extreme multi-label classification~(XMC) task involves learning a classifier that can predict from a large label set the most relevant subset of labels for a data instance. While deep neural networks~(DNNs) have demonstrated remarkable success in XMC problems, the task is still challenging because it must deal with a large number of output labels, which make the DNN training computationally expensive. This paper addresses the issue by exploring the use of random circular vectors, where each vector component is represented as a complex amplitude. In our framework, we can develop an output layer and loss function of DNNs for XMC by representing the final output layer as a fully connected layer that directly predicts a low-dimensional circular vector encoding a set of labels for a data instance. We conducted experiments on synthetic datasets to verify that circular vectors have better label encoding capacity and retrieval ability than normal real-valued vectors. Then, we conducted experiments on actual XMC datasets and found that these appealing properties of circular vectors contribute to significant improvements in task performance compared with a previous model using random real-valued vectors, while reducing the size of the output layers by up to 99%.