Bayesian Partial Reduced-Rank Regression

TL;DR

This paper introduces a Bayesian Partial Reduced-Rank Regression method that simultaneously infers group structures and ranks in data, providing a flexible approach to uncover hidden relationships with uncertainty quantification.

Contribution

It proposes a novel Bayesian framework that estimates unknown group memberships and ranks in reduced-rank regression, addressing limitations of existing methods.

Findings

Successfully applied to synthetic data demonstrating accurate group and rank recovery.

Effective on real-world data revealing meaningful hidden structures.

Provides full uncertainty quantification for inferred parameters.

Abstract

Reduced-rank (RR) regression may be interpreted as a dimensionality reduction technique able to reveal complex relationships among the data parsimoniously. However, RR regression models typically overlook any potential group structure among the responses by assuming a low-rank structure on the coefficient matrix. To address this limitation, a Bayesian Partial RR (BPRR) regression is exploited, where the response vector and the coefficient matrix are partitioned into low- and full-rank sub-groups. As opposed to the literature, which assumes known group structure and rank, a novel strategy is introduced that treats them as unknown parameters to be estimated. The main contribution is two-fold: an approach to infer the low- and full-rank group memberships from the data is proposed, and then, conditionally on this allocation, the corresponding (reduced) rank is estimated. Both steps are carried out in a Bayesian approach, allowing for full uncertainty quantification and based on a partially collapsed Gibbs sampler. It relies on a Laplace approximation of the marginal likelihood and the Metropolized Shotgun Stochastic Search to estimate the group allocation efficiently. Applications to synthetic and real-world data reveal the potential of the proposed method to reveal hidden structures in the data.