Non-Concave Penalization in Linear Mixed-Effects Models and Regularized Selection of Fixed Effects

TL;DR

This paper develops a method for selecting important fixed effects in linear mixed-effects models using non-concave penalties, with theoretical guarantees and an efficient algorithm, applicable even in high-dimensional settings.

Contribution

It introduces a novel regularized variable selection approach with oracle properties for fixed effects in mixed models, handling high-dimensional data with non-concave penalties.

Findings

Proves asymptotic and variable selection consistency.

Provides an efficient computational algorithm.

Demonstrates applicability beyond mixed models to other non-convex problems.

Abstract

Mixed-effect models are very popular for analyzing data with a hierarchical structure, e.g. repeated observations within subjects in a longitudinal design, patients nested within centers in a multicenter design. However, recently, due to the medical advances, the number of fixed effect covariates collected from each patient can be quite large, e.g. data on gene expressions of each patient, and all of these variables are not necessarily important for the outcome. So, it is very important to choose the relevant covariates correctly for obtaining the optimal inference for the overall study. On the other hand, the relevant random effects will often be low-dimensional and pre-specified. In this paper, we consider regularized selection of important fixed effect variables in linear mixed-effects models along with maximum penalized likelihood estimation of both fixed and random effect parameters based on general non-concave penalties. Asymptotic and variable selection consistency with oracle properties are proved for low-dimensional cases as well as for high-dimensionality of non-polynomial order of sample size (number of parameters is much larger than sample size). We also provide a suitable computationally efficient algorithm for implementation. Additionally, all the theoretical results are proved for a general non-convex optimization problem that applies to several important situations well beyond the mixed model set-up (like finite mixture of regressions etc.) illustrating the huge range of applicability of our proposal.