A Hierarchical Convex Optimization for Multiclass SVM Achieving Maximum Pairwise Margins with Least Empirical Hinge-Loss

TL;DR

This paper introduces a novel hierarchical convex optimization framework for multiclass SVMs that maximizes pairwise margins while minimizing empirical hinge-loss, extending previous models with a fixed point approach.

Contribution

It presents the first robust, faithful multiclass SVM optimization model using hierarchical convex optimization and fixed point theory, incorporating generalized hinge loss.

Findings

Applicable hybrid steepest descent method for complex hierarchical optimization

Achieves maximum pairwise margins with minimal hinge-loss in multiclass SVMs

Extends fixed point theory to new class of optimization problems

Abstract

In this paper, we formulate newly a hierarchical convex optimization for multiclass SVM achieving maximum pairwise margins with least empirical hinge-loss. This optimization problem is a most faithful as well as robust multiclass extension of an NP-hard hierarchical optimization appeared for the first time in the seminal paper by C.~Cortes and V.~Vapnik almost 25 years ago. By extending the very recent fixed point theoretic idea [Yamada-Yamagishi 2019] with the generalized hinge loss function [Crammer-Singer 2001], we show that the hybrid steepest descent method [Yamada 2001] in the computational fixed point theory is applicable to this much more complex hierarchical convex optimization problem.