Fast algorithms for large scale generalized distance weighted discrimination

TL;DR

This paper introduces a scalable and robust algorithm for large-scale generalized distance weighted discrimination, improving classification in high-dimensional, low-sample-size data scenarios and outperforming existing SVM solvers in efficiency.

Contribution

The paper develops a new scalable algorithm for generalized DWD that handles large datasets efficiently, surpassing existing methods in speed and robustness.

Findings

Algorithm is highly efficient on real datasets.

Outperforms LIBLINEAR and LIBSVM in large-scale problems.

Effective in high-dimensional, low-sample-size settings.

Abstract

High dimension low sample size statistical analysis is important in a wide range of applications. In such situations, the highly appealing discrimination method, support vector machine, can be improved to alleviate data piling at the margin. This leads naturally to the development of distance weighted discrimination (DWD), which can be modeled as a second-order cone programming problem and solved by interior-point methods when the scale (in sample size and feature dimension) of the data is moderate. Here, we design a scalable and robust algorithm for solving large scale generalized DWD problems. Numerical experiments on real data sets from the UCI repository demonstrate that our algorithm is highly efficient in solving large scale problems, and sometimes even more efficient than the highly optimized LIBLINEAR and LIBSVM for solving the corresponding SVM problems.