Solution Path Algorithm for Twin Multi-class Support Vector Machine

TL;DR

This paper introduces a fast regularization parameter tuning algorithm for twin multi-class support vector machines, enabling efficient multi-class classification without solving quadratic programming problems.

Contribution

It proposes a novel sample data partition strategy and a piecewise linear property of Lagrangian multipliers to accelerate parameter tuning in twin multi-class SVMs.

Findings

Achieves comparable classification performance to existing methods.

Eliminates the need to solve quadratic programming problems.

Demonstrates effectiveness on nine UCI datasets.

Abstract

The twin support vector machine and its extensions have made great achievements in dealing with binary classification problems. However, it suffers from difficulties in effective solution of multi-classification and fast model selection. This work devotes to the fast regularization parameter tuning algorithm for the twin multi-class support vector machine. Specifically, a novel sample data set partition strategy is first adopted, which is the basis for the model construction. Then, combining the linear equations and block matrix theory, the Lagrangian multipliers are proved to be piecewise linear w.r.t. the regularization parameters, so that the regularization parameters are continuously updated by only solving the break points. Next, Lagrangian multipliers are proved to be 1 as the regularization parameter approaches infinity, thus, a simple yet effective initialization algorithm is devised. Finally, eight kinds of events are defined to seek for the starting event for the next iteration. Extensive experimental results on nine UCI data sets show that the proposed method can achieve comparable classification performance without solving any quadratic programming problem.