Conditional Sparse Coding and Grouped Multivariate Regression

TL;DR

This paper introduces a conditional sparse coding method for multivariate regression with grouped data, capturing shared and group-specific structures, and demonstrates its effectiveness through simulations and brain imaging experiments.

Contribution

It proposes a novel conditional sparse coding approach for grouped multivariate regression, combining dictionary learning with sparsity to improve predictive accuracy.

Findings

Outperforms reduced rank regression in simulations

Effective in brain imaging data analysis

Provides theoretical guarantees on predictive accuracy

Abstract

We study the problem of multivariate regression where the data are naturally grouped, and a regression matrix is to be estimated for each group. We propose an approach in which a dictionary of low rank parameter matrices is estimated across groups, and a sparse linear combination of the dictionary elements is estimated to form a model within each group. We refer to the method as conditional sparse coding since it is a coding procedure for the response vectors Y conditioned on the covariate vectors X. This approach captures the shared information across the groups while adapting to the structure within each group. It exploits the same intuition behind sparse coding that has been successfully developed in computer vision and computational neuroscience. We propose an algorithm for conditional sparse coding, analyze its theoretical properties in terms of predictive accuracy, and present the results of simulation and brain imaging experiments that compare the new technique to reduced rank regression.