Fast projections onto mixed-norm balls with applications

TL;DR

This paper introduces fast batch and online algorithms for projecting onto mixed-norm balls, enabling scalable solutions for structured sparsity models like multitask lasso.

Contribution

It provides the first scalable algorithms for projections onto mixed-norm balls, applicable to joint sparsity models such as multitask lasso.

Findings

Developed efficient batch projection algorithm

Designed stochastic-gradient projection method

Applied methods to multitask lasso

Abstract

Joint sparsity offers powerful structural cues for feature selection, especially for variables that are expected to demonstrate a "grouped" behavior. Such behavior is commonly modeled via group-lasso, multitask lasso, and related methods where feature selection is effected via mixed-norms. Several mixed-norm based sparse models have received substantial attention, and for some cases efficient algorithms are also available. Surprisingly, several constrained sparse models seem to be lacking scalable algorithms. We address this deficiency by presenting batch and online (stochastic-gradient) optimization methods, both of which rely on efficient projections onto mixed-norm balls. We illustrate our methods by applying them to the multitask lasso. We conclude by mentioning some open problems.