Numerical optimization for the compatibility constant of the lasso

TL;DR

This paper introduces a numerical method to compute the compatibility constant for the lasso, reducing the problem to quadratic programming and enabling analysis in high-dimensional settings.

Contribution

It presents a novel quadratic programming approach to efficiently compute the compatibility constant given the support of true coefficients.

Findings

The quadratic programming approach simplifies computation of the compatibility constant.

The method performs well in finite-sample simulations under various parameters.

Application to real data demonstrates practical utility of the approach.

Abstract

The compatibility constant plays an important role in evaluating the prediction error of the lasso in high-dimensional settings. However, the computation of the compatibility constant is generally difficult because it is a complicated nonconvex optimization problem. In this study, we present a numerical approach to compute the compatibility constant when the support of true regression coefficients is given. We show that the optimization problem reduces to a quadratic programming (QP) once the signs of the nonzero coefficients are specified. In this case, the compatibility constant can be obtained by solving QPs for all possible sign combinations. We also formulate a mixed-integer QP (MIQP) approach that can be applied when the number of true nonzero coefficients is large. We investigate the finite-sample behavior of the compatibility constant for simulated data under various parameter settings and compare the prediction error with its theoretical upper bound. The behavior of the compatibility constant in finite samples is also investigated through a real data analysis.