Principal component analysis for Gaussian process posteriors

TL;DR

This paper introduces GP-PCA, a method extending principal component analysis to Gaussian process posteriors, enabling low-dimensional representations for meta-learning within an information geometrical framework.

Contribution

It proposes GP-PCA for low-dimensional GP posterior spaces, reducing infinite-dimensional problems to finite dimensions using information geometry, and demonstrates its effectiveness via variational inference.

Findings

Effective low-dimensional representation of GP posteriors

Improved meta-learning performance

Validation through experiments

Abstract

This paper proposes an extension of principal component analysis for Gaussian process (GP) posteriors, denoted by GP-PCA. Since GP-PCA estimates a low-dimensional space of GP posteriors, it can be used for meta-learning, which is a framework for improving the performance of target tasks by estimating a structure of a set of tasks. The issue is how to define a structure of a set of GPs with an infinite-dimensional parameter, such as coordinate system and a divergence. In this study, we reduce the infiniteness of GP to the finite-dimensional case under the information geometrical framework by considering a space of GP posteriors that have the same prior. In addition, we propose an approximation method of GP-PCA based on variational inference and demonstrate the effectiveness of GP-PCA as meta-learning through experiments.