DECAF: Deep Extreme Classification with Label Features

TL;DR

DECAF is a deep learning-based extreme multi-label classification algorithm that leverages label metadata to improve accuracy and speed, enabling real-time predictions on datasets with millions of labels.

Contribution

It introduces a novel model architecture and training method that effectively incorporates label metadata, achieving higher accuracy and faster inference in extreme classification tasks.

Findings

DECAF outperforms leading classifiers by 2-6% in accuracy.

DECAF is up to 22 times faster at inference.

It is suitable for real-time applications with millions of labels.

Abstract

Extreme multi-label classification (XML) involves tagging a data point with its most relevant subset of labels from an extremely large label set, with several applications such as product-to-product recommendation with millions of products. Although leading XML algorithms scale to millions of labels, they largely ignore label meta-data such as textual descriptions of the labels. On the other hand, classical techniques that can utilize label metadata via representation learning using deep networks struggle in extreme settings. This paper develops the DECAF algorithm that addresses these challenges by learning models enriched by label metadata that jointly learn model parameters and feature representations using deep networks and offer accurate classification at the scale of millions of labels. DECAF makes specific contributions to model architecture design, initialization, and training, enabling it to offer up to 2-6% more accurate prediction than leading extreme classifiers on publicly available benchmark product-to-product recommendation datasets, such as LF-AmazonTitles-1.3M. At the same time, DECAF was found to be up to 22x faster at inference than leading deep extreme classifiers, which makes it suitable for real-time applications that require predictions within a few milliseconds. The code for DECAF is available at the following URL https://github.com/Extreme-classification/DECAF.