Nonconvex penalized multitask regression using data depth-based penalties

TL;DR

This paper introduces a novel nonconvex penalty based on data depth functions for multitask sparse regression, offering theoretical guarantees and practical effectiveness in simulations and real data.

Contribution

It develops a new data depth-based penalty for multitask regression, with theoretical properties and an efficient algorithm, outperforming existing methods.

Findings

Near-minimax optimal risk for orthogonal design

Effective in simulations and real data

Outperforms some existing sparse regression methods

Abstract

We propose a new class of nonconvex penalty functions, based on data depth functions, for multitask sparse penalized regression. These penalties quantify the relative position of rows of the coefficient matrix from a fixed distribution centered at the origin. We derive the theoretical properties of an approximate one-step sparse estimator of the coefficient matrix using local linear approximation of the penalty function, and provide algorithm for its computation. For orthogonal design and independent responses, the resulting thresholding rule enjoys near-minimax optimal risk performance, similar to the adaptive lasso (Zou, 2006). A simulation study and real data analysis demonstrate its effectiveness compared to some of the present methods that provide sparse solutions in multivariate regression.