Lagrange Multipliers and Duality with Applications to Constrained Support Vector Machine

TL;DR

This paper advances the theoretical understanding of Lagrange multipliers and duality in nonsmooth convex optimization, and introduces a generalized SVM model with new regularization techniques analyzed through duality and optimization algorithms.

Contribution

It extends duality theory to nonsmooth convex problems in Hilbert spaces and proposes a novel regularized SVM model with comprehensive theoretical and numerical analysis.

Findings

Established strong Lagrangian duality for nonsmooth convex problems

Developed a new generalized SVM model with geometric regularizer

Designed a subgradient and primal-dual algorithms for the new SVM

Abstract

In this paper, we employ the concept of quasi-relative interior to analyze the method of Lagrange multipliers and establish strong Lagrangian duality for nonsmooth convex optimization problems in Hilbert spaces. Then, we generalize the classical support vector machine (SVM) model by incorporating a new geometric constraint or a regularizer on the separating hyperplane, serving as a regularization mechanism for the SVM model. This new SVM model is examined using Lagrangian duality and other convex optimization techniques in both theoretical and numerical aspects via a new subgradient algorithm as well as a primal-dual method.