An Exact Solver for the Weston-Watkins SVM Subproblem

TL;DR

This paper introduces an exact solver for the Weston-Watkins SVM subproblem, leveraging a new reparametrization, leading to faster solutions and linear convergence proofs for multiclass classification tasks.

Contribution

It presents a novel exact solver for the Weston-Watkins SVM subproblem, improving speed and providing convergence guarantees.

Findings

Significant speed-up over existing solvers for large class numbers

Proves linear convergence of the overall solver

Effective for linear Weston-Watkins SVMs

Abstract

Recent empirical evidence suggests that the Weston-Watkins support vector machine is among the best performing multiclass extensions of the binary SVM. Current state-of-the-art solvers repeatedly solve a particular subproblem approximately using an iterative strategy. In this work, we propose an algorithm that solves the subproblem exactly using a novel reparametrization of the Weston-Watkins dual problem. For linear WW-SVMs, our solver shows significant speed-up over the state-of-the-art solver when the number of classes is large. Our exact subproblem solver also allows us to prove linear convergence of the overall solver.