Nonconvex Penalization in Sparse Estimation: An Approach Based on the Bernstein Function

TL;DR

This paper introduces a novel class of nonconvex penalties for sparse estimation based on Bernstein functions, providing theoretical properties, algorithms, and empirical validation for high-dimensional regression and classification tasks.

Contribution

It develops a new nonconvex penalty framework using Bernstein functions, along with tailored algorithms and convergence analysis for sparse high-dimensional estimation.

Findings

Effective thresholding function derived from Bernstein penalties

Coordinate descent algorithm suitable for regression with Bernstein penalties

Empirical validation of Bernstein penalties in high-dimensional problems

Abstract

In this paper we study nonconvex penalization using Bernstein functions whose first-order derivatives are completely monotone. The Bernstein function can induce a class of nonconvex penalty functions for high-dimensional sparse estimation problems. We derive a thresholding function based on the Bernstein penalty and discuss some important mathematical properties in sparsity modeling. We show that a coordinate descent algorithm is especially appropriate for regression problems penalized by the Bernstein function. We also consider the application of the Bernstein penalty in classification problems and devise a proximal alternating linearized minimization method. Based on theory of the Kurdyka-Lojasiewicz inequality, we conduct convergence analysis of these alternating iteration procedures. We particularly exemplify a family of Bernstein nonconvex penalties based on a generalized Gamma measure and conduct empirical analysis for this family.