A new sparsity promoting residual transform operator for Lasso regression

TL;DR

This paper introduces a novel residual transform operator for Lasso regression that adaptively promotes sparsity in signals with varying piecewise smoothness, overcoming limitations of fixed variability assumptions.

Contribution

It proposes a residual transform operator that reduces variability-dependent errors without prior knowledge of the signal's variability, enhancing Lasso's flexibility.

Findings

Effective reduction of variability-dependent errors.

Applicable to signals with unknown or varying piecewise behavior.

Improves sparsity promotion in complex signal environments.

Abstract

Lasso regression is a widely employed approach within the $\ell_1$ regularization framework used to promote sparsity and recover piecewise smooth signals $f:[a,b) \rightarrow \mathbb{R}$ when the given observations are obtained from noisy, blurred, and/or incomplete data environments. In choosing the regularizing sparsity-promoting operator, it is assumed that the particular type of variability of the underlying signal, for example, piecewise constant or piecewise linear behavior across the entire domain, is both known and fixed. Such an assumption is problematic in more general cases, e.g.~when a signal exhibits piecewise oscillatory behavior with varying wavelengths and magnitudes. To address the limitations of assuming a fixed (and typically low order) variability when choosing a sparsity-promoting operator, this investigation proposes a novel residual transform operator that can be used within the Lasso regression formulation. In a nutshell, the idea is that for a general piecewise smooth signal $f$, it is possible to design two operators $\mathcal L_1$ and $\mathcal L_2$ such that $\mathcal L_1{\boldsymbol f} \approx \mathcal L_2{\boldsymbol f}$, where ${\boldsymbol f} \in \mathbb{R}^n$ is a discretized approximation of $f$, but $\mathcal L_1 \not\approx \mathcal L_2$. The corresponding residual transform operator, $\mathcal L = \mathcal L_1- \mathcal L_2$, yields a result that (1) effectively reduces the variability dependent error that occurs when applying either $\mathcal L_1$ or $\mathcal L_2$ to ${\boldsymbol f}$, a property that holds even when $\mathcal L_1{\boldsymbol f} \approx \mathcal L_2{\boldsymbol f}$ is not a good approximation to the true sparse domain vector of ${\boldsymbol f}$, and (2) does not require $\mathcal L_1$ or $\mathcal L_2$ to have prior information regarding the variability of the underlying signal.