Randomized Algorithms for Large scale SVMs

TL;DR

This paper introduces RandSVM, a randomized algorithm that efficiently trains large-scale SVMs using random projections, reducing computational complexity while maintaining accuracy on diverse datasets.

Contribution

The paper presents a novel randomized algorithm for large-scale SVM training that leverages random projections to reduce problem size and computational effort.

Findings

Scales up existing SVM learners without accuracy loss

Uses random projections to estimate combinatorial dimension as O(log n)

Demonstrates effectiveness on synthetic and real datasets

Abstract

We propose a randomized algorithm for training Support vector machines(SVMs) on large datasets. By using ideas from Random projections we show that the combinatorial dimension of SVMs is $O({log} n)$ with high probability. This estimate of combinatorial dimension is used to derive an iterative algorithm, called RandSVM, which at each step calls an existing solver to train SVMs on a randomly chosen subset of size $O({log} n)$. The algorithm has probabilistic guarantees and is capable of training SVMs with Kernels for both classification and regression problems. Experiments done on synthetic and real life data sets demonstrate that the algorithm scales up existing SVM learners, without loss of accuracy.