Kinetic Energy Plus Penalty Functions for Sparse Estimation

TL;DR

This paper introduces a new family of sparsity-inducing penalty functions called Kinetic Energy Plus (KEP), inspired by special relativity, with proven properties and efficient algorithms for high-dimensional sparse estimation.

Contribution

The paper proposes the KEP penalty functions, analyzes their mathematical properties, develops a thresholding operator, and demonstrates their effectiveness in high-dimensional sparse modeling.

Findings

KEP functions effectively induce sparsity in high-dimensional data.

A coordinate descent algorithm is suitable for optimizing KEP-based models.

Theoretical and empirical results show KEP's efficiency and effectiveness.

Abstract

In this paper we propose and study a family of sparsity-inducing penalty functions. Since the penalty functions are related to the kinetic energy in special relativity, we call them \emph{kinetic energy plus} (KEP) functions. We construct the KEP function by using the concave conjugate of a $\chi^2$-distance function and present several novel insights into the KEP function with $q=1$. In particular, we derive a thresholding operator based on the KEP function, and prove its mathematical properties and asymptotic properties in sparsity modeling. Moreover, we show that a coordinate descent algorithm is especially appropriate for the KEP function. Additionally, we discuss the relationship of KEP with the penalty functions $\ell_{1/2}$ and MCP. The theoretical and empirical analysis validates that the KEP function is effective and efficient in high-dimensional data modeling.