Hierarchical Convex Optimization by the Hybrid Steepest Descent Method with Proximal Splitting Operators -- Enhancements of SVM and Lasso

TL;DR

This paper introduces new algorithmic strategies for hierarchical convex optimization using hybrid steepest descent and proximal splitting, with applications to enhancing SVM and Lasso methods.

Contribution

It develops practical algorithms for hierarchical convex problems by applying hybrid steepest descent to nonexpansive operators, advancing proximal splitting techniques.

Findings

Algorithms effectively solve hierarchical convex optimization problems.

Applications improve SVM and Lasso performance.

Proximal splitting methods enable strategic solution selection.

Abstract

The breakthrough ideas in the modern proximal splitting methodologies allow us to express the set of all minimizers of a superposition of multiple nonsmooth convex functions as the fixed point set of computable nonexpansive operators. In this paper, we present practical algorithmic strategies for the hierarchical convex optimization problems which require further strategic selection of a most desirable vector from the solution set of the standard convex optimization. The proposed algorithms are established by applying the hybrid steepest descent method to special nonexpansive operators designed through the art of proximal splitting. We also present applications of the proposed strategies to certain unexplored hierarchical enhancements of the support vector machine and the Lasso estimator.