# A Semismooth Newton Method for Support Vector Classification and Regression

## TL;DR

This paper introduces a semismooth Newton method tailored for support vector machine models, achieving fast convergence and reduced computational complexity, especially effective for large-scale datasets.

## Contribution

It explores the sparse structure of SVM models to significantly lower computational complexity while maintaining quadratic convergence, outperforming existing solvers on large datasets.

## Key findings

- The method converges quadratically and is computationally efficient.
- It outperforms leading solvers like DCD and TRON on large-scale problems.
- It solves large SVM problems in seconds, demonstrating practical efficiency.

## Abstract

Support vector machine is an important and fundamental technique in machine learning. In this paper, we apply a semismooth Newton method to solve two typical SVM models: the L2-loss SVC model and the \epsilon-L2-loss SVR model. The semismooth Newton method is widely used in optimization community. A common belief on the semismooth Newton method is its fast convergence rate as well as high computational complexity. Our contribution in this paper is that by exploring the sparse structure of the models, we significantly reduce the computational complexity, meanwhile keeping the quadratic convergence rate. Extensive numerical experiments demonstrate the outstanding performance of the semismooth Newton method, especially for problems with huge size of sample data (for news20.binary problem with 19996 features and 1355191 samples, it only takes three seconds). In particular, for the \epsilon-L2-loss SVR model, the semismooth Newton method significantly outperforms the leading solvers including DCD and TRON.