On the primal-dual dynamics of Support Vector Machines

TL;DR

This paper models the primal-dual optimization process of Support Vector Machines as a dynamical system and proves its global convergence to the optimal solution using passivity theory.

Contribution

It introduces a dynamical system framework for SVM optimization and demonstrates its global asymptotic stability, a novel approach in SVM analysis.

Findings

The dynamical system converges to the SVM optimal solution from any initial condition.

Passivity theory is effective in proving the stability of the SVM primal-dual dynamics.

Simulations confirm the theoretical convergence results.

Abstract

The aim of this paper is to study the convergence of the primal-dual dynamics pertaining to Support Vector Machines (SVM). The optimization routine, used for determining an SVM for classification, is first formulated as a dynamical system. The dynamical system is constructed such that its equilibrium point is the solution to the SVM optimization problem. It is then shown, using passivity theory, that the dynamical system is global asymptotically stable. In other words, the dynamical system converges onto the optimal solution asymptotically, irrespective of the initial condition. Simulations and computations are provided for corroboration.