CoMET: A Compressed Bayesian Mixed-Effects Model for High-Dimensional Tensors

TL;DR

CoMET introduces a Bayesian tensor mixed-effects model that efficiently handles high-dimensional tensor covariates, enabling structured random-effects modeling, fixed-effects selection, and scalable computation with theoretical guarantees.

Contribution

This paper presents CoMET, a novel Bayesian mixed-effects model for high-dimensional tensor data, incorporating structured random-effects and fixed-effects selection with efficient computation and theoretical validation.

Findings

Outperforms penalized methods in simulations.

Achieves linear computational complexity with tensor dimensions.

Provides theoretical guarantees for posterior risk decay.

Abstract

Mixed-effects models are fundamental tools for analyzing clustered and repeated-measures data, but existing high-dimensional methods largely focus on penalized estimation with vector-valued covariates. Bayesian alternatives in this regime are limited, with no sampling-based mixed-effects framework that supports tensor-valued fixed- and random-effects covariates while remaining computationally tractable. We propose the Compressed Mixed-Effects Tensor (CoMET) model for high-dimensional repeated-measures data with scalar responses and tensor-valued covariates. CoMET performs structured, mode-wise random projection of the random-effects covariance, yielding a low-dimensional covariance parameter that admits simple Gaussian prior specification and enables efficient imputation of compressed random-effects. For the mean structure, CoMET leverages a low-rank tensor decomposition and margin-structured Horseshoe priors to enable fixed-effects selection. These design choices lead to an efficient collapsed Gibbs sampler whose computational complexity grows approximately linearly with the tensor covariate dimensions. We establish high-dimensional theoretical guarantees by identifying regularity conditions under which CoMET's posterior predictive risk decays to zero. Empirically, CoMET outperforms penalized competitors across a range of simulation studies and two benchmark applications involving facial-expression prediction and music emotion modeling.