On Sparsity Inducing Regularization Methods for Machine Learning

TL;DR

This paper reviews a broad class of sparsity regularization methods in machine learning, proposing a unified approach for solving these problems and discussing applications to support vector machines.

Contribution

It introduces a general framework for solving convex regularization problems involving sparsity, applicable to various methods like group Lasso and Fused Lasso, using proximity operators.

Findings

Unified approach for sparsity regularization problems

Application to support vector machines

Enhanced solution techniques for convex regularizers

Abstract

During the past years there has been an explosion of interest in learning methods based on sparsity regularization. In this paper, we discuss a general class of such methods, in which the regularizer can be expressed as the composition of a convex function $\omega$ with a linear function. This setting includes several methods such the group Lasso, the Fused Lasso, multi-task learning and many more. We present a general approach for solving regularization problems of this kind, under the assumption that the proximity operator of the function $\omega$ is available. Furthermore, we comment on the application of this approach to support vector machines, a technique pioneered by the groundbreaking work of Vladimir Vapnik.