Scaling Up ROC-Optimizing Support Vector Machines

TL;DR

This paper introduces a scalable ROC-optimizing SVM that reduces computational costs using incomplete U-statistics and low-rank kernel approximations, maintaining performance while significantly speeding up training.

Contribution

It develops a computationally efficient variant of ROC-SVM using incomplete U-statistics and kernel approximation, enabling practical nonlinear classification.

Findings

Achieves comparable AUC to original ROC-SVM

Reduces training time significantly

Validates effectiveness on synthetic and real datasets

Abstract

The ROC-SVM, originally proposed by Rakotomamonjy, directly maximizes the area under the ROC curve (AUC) and has become an attractive alternative of the conventional binary classification under the presence of class imbalance. However, its practical use is limited by high computational cost, as training involves evaluating all $O(n^2)$. To overcome this limitation, we develop a scalable variant of the ROC-SVM that leverages incomplete U-statistics, thereby substantially reducing computational complexity. We further extend the framework to nonlinear classification through a low-rank kernel approximation, enabling efficient training in reproducing kernel Hilbert spaces. Theoretical analysis establishes an error bound that justifies the proposed approximation, and empirical results on both synthetic and real datasets demonstrate that the proposed method achieves comparable AUC performance to the original ROC-SVM with drastically reduced training time.