Low-Rank Plus Sparse Matrix Transfer Learning under Growing Representations and Ambient Dimensions

TL;DR

This paper introduces a transfer learning framework for structured matrix estimation that accounts for growing ambient dimensions and representations, providing theoretical guarantees and empirical validation.

Contribution

It proposes a novel transfer framework with an anchored alternating projection estimator that effectively separates source, low-rank, and sparse components in expanding spaces.

Findings

The estimator achieves improved error bounds when rank and sparsity increments are small.

The framework applies to Markov transition matrix estimation with dependent noise.

Empirical results validate the transfer gains in structured covariance estimation.

Abstract

Learning systems often expand their ambient features or latent representations over time, embedding earlier representations into larger spaces with limited new latent structure. We study transfer learning for structured matrix estimation under simultaneous growth of the ambient dimension and the intrinsic representation, where a well-estimated source task is embedded as a subspace of a higher-dimensional target task. We propose a general transfer framework in which the target parameter decomposes into an embedded source component, low-dimensional low-rank innovations, and sparse edits, and develop an anchored alternating projection estimator that preserves transferred subspaces while estimating only low-dimensional innovations and sparse modifications. We establish deterministic error bounds that separate target noise, representation growth, and source estimation error, yielding strictly improved rates when rank and sparsity increments are small. We demonstrate the generality of the framework by applying it to two canonical problems. For Markov transition matrix estimation from a single trajectory, we derive end-to-end theoretical guarantees under dependent noise. For structured covariance estimation under enlarged dimensions, we provide complementary theoretical analysis in the appendix and empirically validate consistent transfer gains.