Adaptive scaling for soft-thresholding estimator

TL;DR

This paper introduces an adaptive scaling method for soft-thresholding estimators in wavelet denoising, allowing independent control of thresholding and shrinkage to improve sparsity and prediction accuracy.

Contribution

It proposes a novel component-wise, data-dependent adaptive scaling technique for soft-thresholding, along with a risk-based model selection criterion, enhancing sparse modeling in non-parametric regression.

Findings

The adaptive scaling reduces estimation bias and improves sparsity.

The model selection criterion effectively identifies low-risk models.

Numerical experiments confirm improved denoising performance.

Abstract

Soft-thresholding is a sparse modeling method that is typically applied to wavelet denoising in statistical signal processing and analysis. It has a single parameter that controls a threshold level on wavelet coefficients and, simultaneously, amount of shrinkage for coefficients of un-removed components. This parametrization is possible to cause excess shrinkage, thus, estimation bias at a sparse representation; i.e. there is a dilemma between sparsity and prediction accuracy. To relax this problem, we considered to introduce positive scaling on soft-thresholding estimator, by which threshold level and amount of shrinkage are independently controlled. Especially, in this paper, we proposed component-wise and data-dependent scaling in a setting of non-parametric orthogonal regression problem including discrete wavelet transform. We call our scaling method adaptive scaling. We here employed soft-thresholding method based on LARS(least angle regression), by which the model selection problem reduces to the determination of the number of un-removed components. We derived a risk under LARS-based soft-thresholding with the proposed adaptive scaling and established a model selection criterion as an unbiased estimate of the risk. We also analyzed some properties of the risk curve and found that the model selection criterion is possible to select a model with low risk and high sparsity compared to a naive soft-thresholding method. This theoretical speculation was verified by a simple numerical experiment and an application to wavelet denoising.