TL-PCA: Transfer Learning of Principal Component Analysis

TL;DR

TL-PCA introduces a transfer learning method for PCA that leverages source task knowledge to improve dimensionality reduction in target tasks with limited data, outperforming standard PCA.

Contribution

The paper proposes a novel transfer learning approach for PCA that incorporates source task information via a penalty term, enabling better subspace estimation with limited target data.

Findings

Improved data representation in image datasets using TL-PCA.

TL-PCA's eigenvector count is not limited by target data size.

Enhanced dimensionality reduction performance over standard PCA.

Abstract

Principal component analysis (PCA) can be significantly limited when there is too few examples of the target data of interest. We propose a transfer learning approach to PCA (TL-PCA) where knowledge from a related source task is used in addition to the scarce data of a target task. Our TL-PCA has two versions, one that uses a pretrained PCA solution of the source task, and another that uses the source data. Our proposed approach extends the PCA optimization objective with a penalty on the proximity of the target subspace and the source subspace as given by the pretrained source model or the source data. This optimization is solved by eigendecomposition for which the number of data-dependent eigenvectors (i.e., principal directions of TL-PCA) is not limited to the number of target data examples, which is a root cause that limits the standard PCA performance. Accordingly, our results for image datasets show that the representation of test data is improved by TL-PCA for dimensionality reduction where the learned subspace dimension is lower or higher than the number of target data examples.