Multipurpose Lasso

TL;DR

This paper introduces a new differentiable approximation to lasso that maintains sparsity and smoothness, enabling more accurate and flexible regularization in high-dimensional and low-sample scenarios.

Contribution

It proposes a novel differentiable lasso approximation that retains the benefits of lasso and ridge, with proven theoretical properties and broad applicability.

Findings

Performs comparably to lasso and ridge in simulations

Maintains sparsity with differentiability

Applicable with standard convex optimization methods

Abstract

Nowadays, l1 penalized likelihood has absorbed a high amount of consideration due to its simplicity and well developed theoretical properties. This method is known as a reliable method in order to apply in a broad range of applications including high-dimensional cases. On the other hand, $L_1$ driven methods, precisely lasso dependent regularizations, suffer the loss of sparsity when the number of observations is too low. In this paper we address a new differentiable approximation of lasso that can produce the same results as lasso and ridge and also can produce smooth results. We prove the theoretical properties of the model as well as its computation complexity. Due to differentiability, proposed method can be implemented by means of the majority of convex optimization methods in literature. That means a higher accuracy in situations where true coefficients are close to zero that is a major issue of LARS. Simulation study as well as flexibility of the method show that the proposed approach is as reliable as lass and ridge and can be used in both situations.