Model Selection in Linear Mixed Models

TL;DR

This paper reviews recent advances in model selection techniques for linear mixed models, addressing the complexities of covariance structure selection and comparing various methods like information criteria, penalization, Fence, and Bayesian approaches.

Contribution

It provides a comprehensive review and comparison of different model selection methods for linear mixed models, highlighting their properties and interrelationships.

Findings

Comparison of information criteria and penalization methods.

Discussion of computational challenges in covariance selection.

Analysis of Bayesian and Fence procedures.

Abstract

Linear mixed effects models are highly flexible in handling a broad range of data types and are therefore widely used in applications. A key part in the analysis of data is model selection, which often aims to choose a parsimonious model with other desirable properties from a possibly very large set of candidate statistical models. Over the last 5-10 years the literature on model selection in linear mixed models has grown extremely rapidly. The problem is much more complicated than in linear regression because selection on the covariance structure is not straightforward due to computational issues and boundary problems arising from positive semidefinite constraints on covariance matrices. To obtain a better understanding of the available methods, their properties and the relationships between them, we review a large body of literature on linear mixed model selection. We arrange, implement, discuss and compare model selection methods based on four major approaches: information criteria such as AIC or BIC, shrinkage methods based on penalized loss functions such as LASSO, the Fence procedure and Bayesian techniques.