Scalable Multilabel Prediction via Randomized Methods

TL;DR

This paper introduces a scalable approach for multilabel and multiclass prediction that models output dependencies using randomized algorithms for efficient joint prediction without computing all independent predictions.

Contribution

The authors propose a novel regularized nonlinear mapping combined with randomized matrix techniques to efficiently model output dependencies in multilabel classification.

Findings

Achieves state-of-the-art performance on benchmark datasets.

Efficiently models output dependencies without computing all independent predictions.

Applicable to both multilabel and multiclass problems.

Abstract

Modeling the dependence between outputs is a fundamental challenge in multilabel classification. In this work we show that a generic regularized nonlinearity mapping independent predictions to joint predictions is sufficient to achieve state-of-the-art performance on a variety of benchmark problems. Crucially, we compute the joint predictions without ever obtaining any independent predictions, while incorporating low-rank and smoothness regularization. We achieve this by leveraging randomized algorithms for matrix decomposition and kernel approximation. Furthermore, our techniques are applicable to the multiclass setting. We apply our method to a variety of multiclass and multilabel data sets, obtaining state-of-the-art results.