# Improved Probabilistic Principal Component Analysis for Application to Reduced Order Modeling

## TL;DR

This paper introduces an improved probabilistic PCA-based reduced order modeling approach that estimates noise, optimizes model dimension, and enhances projection accuracy for noisy data, advancing stochastic modeling techniques.

## Contribution

The work develops an orthonormal basis generation method using PPCA, incorporates automatic model dimension selection, and introduces a statistical projection method leveraging noise variance estimates.

## Key findings

- Enhanced accuracy in noise-affected data reconstruction.
- Automatic model dimension selection improves model efficiency.
- Better noise estimation leads to more reliable predictions.

## Abstract

In our previous work, a reduced order model (ROM) for a stochastic system was made, where noisy data was projected onto principal component analysis (PCA)-derived basis vectors to obtain an accurate reconstruction of the noise-free data. That work used techniques designed for deterministic data, PCA was used for the basis function generation and $L_2$ projection was used to create the reconstructions. In this work, probabilistic approaches are used. The probabilistic PCA (PPCA) is used to generate the basis, which then allows the noise in the training data to be estimated. PPCA has also been improved so that the derived basis vectors are orthonormal and the variance of the basis expansion coefficients over the training data set can be estimated. The standard approach assumes a unit variance for these coefficients. Based on the results of the PPCA, model selection criteria are applied to automatically choose the dimension of the ROM. In our previous work, a heuristic approach was used to pick the dimension. Lastly, a new statistical approach is used for the projection step where the variance information obtained from the improved PPCA is used as a prior to improve the projection. This gives improved accuracy over $L_2$ projection when the projected data is noisy. In addition, the noise statistics for the projected data are not assumed to be the same as that of the training data, but are estimated in the projection process. The entire approach gives a fully stochastic method for computing a ROM from noisy training data, determining ideal model selection, and projecting noisy test data, thus enabling accurate predictions of noise-free data from data that is dominated by noise.