Exclusive Group Lasso for Structured Variable Selection

TL;DR

This paper introduces an exclusive group lasso method leveraging atomic norms for structured variable selection, enabling efficient support recovery with theoretical guarantees and practical algorithms.

Contribution

It proposes a novel atomic norm-based regularizer for exclusive group sparsity, along with efficient algorithms and asymptotic consistency analysis.

Findings

Supports accurate support recovery in simulations

Algorithms are flexible for different structured sparsity patterns

Theoretical analysis confirms asymptotic consistency

Abstract

A structured variable selection problem is considered in which the covariates, divided into predefined groups, activate according to sparse patterns with few nonzero entries per group. Capitalizing on the concept of atomic norm, a composite norm can be properly designed to promote such exclusive group sparsity patterns. The resulting norm lends itself to efficient and flexible regularized optimization algorithms for support recovery, like the proximal algorithm. Moreover, an active set algorithm is proposed that builds the solution by successively including structure atoms into the estimated support. It is also shown that such an algorithm can be tailored to match more rigid structures than plain exclusive group sparsity. Asymptotic consistency analysis (with both the number of parameters as well as the number of groups growing with the observation size) establishes the effectiveness of the proposed solution in terms of signed support recovery under conventional assumptions. Finally, a set of numerical simulations further corroborates the results.