A Sparse Nonlinear Classifier Design Using AUC Optimization

TL;DR

This paper introduces a scalable, sparse nonlinear classifier optimized for AUC performance, addressing scalability and sparsity issues in existing batch and online methods, with promising results on real-world datasets.

Contribution

It proposes a greedy basis function addition method for sparse AUC-optimized classifiers, improving inference speed and sparsity over kernel-based models.

Findings

Achieves significant sparsity without losing AUC performance

Outperforms kernel rankSVM in speed and sparsity

Effective on real-world imbalanced datasets

Abstract

AUC (Area under the ROC curve) is an important performance measure for applications where the data is highly imbalanced. Learning to maximize AUC performance is thus an important research problem. Using a max-margin based surrogate loss function, AUC optimization problem can be approximated as a pairwise rankSVM learning problem. Batch learning methods for solving the kernelized version of this problem suffer from scalability and may not result in sparse classifiers. Recent years have witnessed an increased interest in the development of online or single-pass online learning algorithms that design a classifier by maximizing the AUC performance. The AUC performance of nonlinear classifiers, designed using online methods, is not comparable with that of nonlinear classifiers designed using batch learning algorithms on many real-world datasets. Motivated by these observations, we design a scalable algorithm for maximizing AUC performance by greedily adding the required number of basis functions into the classifier model. The resulting sparse classifiers perform faster inference. Our experimental results show that the level of sparsity achievable can be order of magnitude smaller than the Kernel RankSVM model without affecting the AUC performance much.