A Short Note of Comparison between Convex and Non-convex Penalized Likelihood

TL;DR

This paper compares convex and non-convex penalized likelihood methods in high-dimensional modeling, discussing their advantages, limitations, and practical considerations for method selection.

Contribution

It provides a comprehensive comparison of convex and non-convex penalties, highlighting their theoretical and practical differences in high-dimensional statistical modeling.

Findings

Convex penalties like LASSO are computationally efficient but can introduce bias.

Non-convex penalties like SCAD and MCP reduce bias and have oracle properties.

The choice of penalty depends on the specific problem context and desired trade-offs.

Abstract

This paper compares convex and non-convex penalized likelihood methods in high-dimensional statistical modeling, focusing on their strengths and limitations. Convex penalties, like LASSO, offer computational efficiency and strong theoretical guarantees but often introduce bias in parameter estimation. Non-convex penalties, such as SCAD and MCP, reduce bias and achieve oracle properties but pose optimization challenges due to non-convexity. The paper highlights key differences in bias-variance trade-offs, computational complexity, and robustness, offering practical guidance for method selection. It concludes that the choice depends on the problem context, balancing accuracy