Revisiting the Vector Space Model: Sparse Weighted Nearest-Neighbor Method for Extreme Multi-Label Classification

TL;DR

This paper introduces a Sparse Weighted Nearest-Neighbor method for extreme multi-label classification that is fast, memory-efficient, and performs comparably or better than state-of-the-art models on large-scale datasets.

Contribution

The paper presents a novel sparse nearest-neighbor approach for XMLC, linking it to a generalized vector space model and demonstrating real-time processing capabilities.

Findings

Achieves real-time processing on large XMLC datasets

Performs comparably or better than SOTA models on large datasets

Uses less storage and runs efficiently with a single thread

Abstract

Machine learning has played an important role in information retrieval (IR) in recent times. In search engines, for example, query keywords are accepted and documents are returned in order of relevance to the given query; this can be cast as a multi-label ranking problem in machine learning. Generally, the number of candidate documents is extremely large (from several thousand to several million); thus, the classifier must handle many labels. This problem is referred to as extreme multi-label classification (XMLC). In this paper, we propose a novel approach to XMLC termed the Sparse Weighted Nearest-Neighbor Method. This technique can be derived as a fast implementation of state-of-the-art (SOTA) one-versus-rest linear classifiers for very sparse datasets. In addition, we show that the classifier can be written as a sparse generalization of a representer theorem with a linear kernel. Furthermore, our method can be viewed as the vector space model used in IR. Finally, we show that the Sparse Weighted Nearest-Neighbor Method can process data points in real time on XMLC datasets with equivalent performance to SOTA models, with a single thread and smaller storage footprint. In particular, our method exhibits superior performance to the SOTA models on a dataset with 3 million labels.