Structured Sparsity and Generalization

TL;DR

This paper introduces a data-dependent generalization bound applicable to various structured sparsity regularization methods, including infinite-dimensional cases like the Lasso and multiple kernel learning, demonstrating competitive results.

Contribution

It provides a unified generalization bound for diverse structured sparsity algorithms, extending to infinite-dimensional settings.

Findings

Bound applies to standard squared-norm regularization, Lasso, group Lasso, and multiple kernel learning.

Results are competitive across different regularization schemes.

The bound is applicable in infinite-dimensional spaces such as Hilbert spaces.

Abstract

We present a data dependent generalization bound for a large class of regularized algorithms which implement structured sparsity constraints. The bound can be applied to standard squared-norm regularization, the Lasso, the group Lasso, some versions of the group Lasso with overlapping groups, multiple kernel learning and other regularization schemes. In all these cases competitive results are obtained. A novel feature of our bound is that it can be applied in an infinite dimensional setting such as the Lasso in a separable Hilbert space or multiple kernel learning with a countable number of kernels.