Supervised PCA: A Multiobjective Approach

TL;DR

This paper introduces a new supervised PCA method that jointly optimizes prediction accuracy and variance explained, outperforming existing methods and enabling a low-rank extension of generalized linear models.

Contribution

It proposes a multiobjective supervised PCA approach that balances prediction error and variance explained, and offers a statistical reformulation for low-rank generalized linear models.

Findings

Outperforms existing SPCA methods in prediction and variance explained

Provides a flexible framework accommodating arbitrary supervised losses

Enables a novel low-rank extension of generalized linear models

Abstract

Methods for supervised principal component analysis (SPCA) aim to incorporate label information into principal component analysis (PCA), so that the extracted features are more useful for a prediction task of interest. Prior work on SPCA has focused primarily on optimizing prediction error, and has neglected the value of maximizing variance explained by the extracted features. We propose a new method for SPCA that addresses both of these objectives jointly, and demonstrate empirically that our approach dominates existing approaches, i.e., outperforms them with respect to both prediction error and variation explained. Our approach accommodates arbitrary supervised learning losses and, through a statistical reformulation, provides a novel low-rank extension of generalized linear models.