Extreme Multi-label Completion for Semantic Document Labelling with Taxonomy-Aware Parallel Learning

TL;DR

This paper introduces TAMLEC, a novel taxonomy-aware multi-task learning approach for extreme multi-label document completion, leveraging hierarchical label structures and parallel feature sharing to improve accuracy, especially in few-shot scenarios.

Contribution

TAMLEC is the first method to combine taxonomy-aware multi-task learning with parallel feature sharing for XMLCo, enhancing performance on large label sets and few-shot tasks.

Findings

TAMLEC outperforms state-of-the-art XMLCo methods on real datasets.

TAMLEC is highly effective in few-shot label prediction scenarios.

The approach leverages hierarchical label structures for improved accuracy.

Abstract

In Extreme Multi Label Completion (XMLCo), the objective is to predict the missing labels of a collection of documents. Together with XML Classification, XMLCo is arguably one of the most challenging document classification tasks, as the very high number of labels (at least ten of thousands) is generally very large compared to the number of available labelled documents in the training dataset. Such a task is often accompanied by a taxonomy that encodes the labels organic relationships, and many methods have been proposed to leverage this hierarchy to improve the results of XMLCo algorithms. In this paper, we propose a new approach to this problem, TAMLEC (Taxonomy-Aware Multi-task Learning for Extreme multi-label Completion). TAMLEC divides the problem into several Taxonomy-Aware Tasks, i.e. subsets of labels adapted to the hierarchical paths of the taxonomy, and trains on these tasks using a dynamic Parallel Feature sharing approach, where some parts of the model are shared between tasks while others are task-specific. Then, at inference time, TAMLEC uses the labels available in a document to infer the appropriate tasks and to predict missing labels. To achieve this result, TAMLEC uses a modified transformer architecture that predicts ordered sequences of labels on a Weak-Semilattice structure that is naturally induced by the tasks. This approach yields multiple advantages. First, our experiments on real-world datasets show that TAMLEC outperforms state-of-the-art methods for various XMLCo problems. Second, TAMLEC is by construction particularly suited for few-shots XML tasks, where new tasks or labels are introduced with only few examples, and extensive evaluations highlight its strong performance compared to existing methods.