Efficient Computing Algorithm for High Dimensional Sparse Support Vector Machine

TL;DR

This paper introduces an efficient, parallelizable ADMM-based algorithm for high-dimensional sparse SVMs, addressing computational challenges posed by non-smooth objectives and ultra-high dimensional data.

Contribution

It presents a novel ADMM algorithm with proven convergence for sparse SVMs, optimized for high-dimensional and parallel computing environments.

Findings

The proposed algorithm converges at a specified rate.

It outperforms existing solvers in high-dimensional settings.

Parallel implementation enhances scalability and efficiency.

Abstract

In recent years, considerable attention has been devoted to the regularization models due to the presence of high-dimensional data in scientific research. Sparse support vector machine (SVM) are useful tools in high-dimensional data analysis, and they have been widely used in the area of econometrics. Nevertheless, the non-smoothness of objective functions and constraints present computational challenges for many existing solvers in the presence of ultra-high dimensional covariates. In this paper, we design efficient and parallelizable algorithms for solving sparse SVM problems with high dimensional data through feature space split. The proposed algorithm is based on the alternating direction method of multiplier (ADMM). We establish the rate of convergence of the proposed ADMM method and compare it with existing solvers in various high and ultra-high dimensional settings. The compatibility of the proposed algorithm with parallel computing can further alleviate the storage and scalability limitations of a single machine in large-scale data processing.