GADGET SVM: A Gossip-bAseD sub-GradiEnT Solver for Linear SVMs

TL;DR

GADGET SVM introduces a distributed gossip-based primal SVM algorithm that efficiently scales to large datasets by enabling local computation and neighbor communication, achieving performance comparable to centralized methods.

Contribution

This paper presents the first distributed gossip-based primal SVM algorithm that operates efficiently on large-scale data with local updates and neighbor communication.

Findings

Performance comparable to centralized SVMs

Scales efficiently with large datasets

Effective in distributed environments

Abstract

In the era of big data, an important weapon in a machine learning researcher's arsenal is a scalable Support Vector Machine (SVM) algorithm. SVMs are extensively used for solving classification problems. Traditional algorithms for learning SVMs often scale super linearly with training set size which becomes infeasible very quickly for large data sets. In recent years, scalable algorithms have been designed which study the primal or dual formulations of the problem. This often suggests a way to decompose the problem and facilitate development of distributed algorithms. In this paper, we present a distributed algorithm for learning linear Support Vector Machines in the primal form for binary classification called Gossip-bAseD sub-GradiEnT (GADGET) SVM. The algorithm is designed such that it can be executed locally on nodes of a distributed system. Each node processes its local homogeneously partitioned data and learns a primal SVM model. It then gossips with random neighbors about the classifier learnt and uses this information to update the model. Extensive theoretical and empirical results suggest that this anytime algorithm has performance comparable to its centralized and online counterparts.