Continuous Relaxation of Discontinuous Shrinkage Operator: Proximal Inclusion and Conversion

TL;DR

This paper introduces a method to derive continuous relaxations of discontinuous shrinkage operators using proximal inclusion and conversion, enabling smoother operators with potential practical benefits.

Contribution

The authors propose a novel framework for converting discontinuous shrinkage operators into continuous, Lipschitz continuous operators through a double inversion process.

Findings

The firm shrinkage operator is derived as a continuous relaxation of the hard shrinkage operator.

A new continuous relaxation operator for ROWL penalty is introduced.

Numerical examples show advantages of the continuous relaxation in practice.

Abstract

We present a principled way of deriving a continuous relaxation of a given discontinuous shrinkage operator, which is based on two fundamental results, proximal inclusion and conversion. Using our results, the discontinuous operator is converted, via double inversion, to a continuous operator; more precisely, the associated ``set-valued'' operator is converted to a ``single-valued'' Lipschitz continuous operator. The first illustrative example is the firm shrinkage operator which can be derived as a continuous relaxation of the hard shrinkage operator. We also derive a new operator as a continuous relaxation of the discontinuous shrinkage operator associated with the so-called reversely ordered weighted L1 (ROWL) penalty. Numerical examples demonstrate potential advantages of the continuous relaxation.