Generative Flexible Latent Structure Regression (GFLSR) model

TL;DR

This paper introduces GFLSR, a generative, flexible latent structure regression model that unifies and extends linear latent variable methods like PLS, enabling model inference, residual analysis, and uncertainty quantification.

Contribution

The paper proposes GFLSR, a novel generative framework for linear latent structure methods, allowing inference and residual analysis, and introduces a specialized Generative-PLS model.

Findings

GFLSR unifies linear latent variable methods under a generative framework.

The recursive structure enables model inference and residual analysis.

A bootstrap algorithm provides uncertainty quantification for parameters and predictions.

Abstract

Latent structure methods, specifically linear continuous latent structure methods, are a type of fundamental statistical learning strategy. They are widely used for dimension reduction, regression and prediction, in the fields of chemometrics, economics, social science and etc. However, due to the lack of model inference, generative form, and unidentifiable parameters, most of these methods are always used as an algorithm, instead of a model. This paper proposed a Generative Flexible Latent Structure Regression (GFLSR) model structure to address this problem. Moreover, we show that most linear continuous latent variable methods can be represented under the proposed framework. The recursive structure allows potential model inference and residual analysis. Then, the traditional Partial Least Squares (PLS) is focused; we show that the PLS can be specialised in the proposed model structure, named Generative-PLS. With a model structure, we analyse the convergence of the parameters and the latent variables. Under additional distribution assumptions, we show that the proposed model structure can lead to model inference without solving the probabilistic model. Additionally, we proposed a novel bootstrap algorithm that enables uncertainty on parameters and on prediction for new datasets. A simulation study and a Real-world dataset are used to verify the proposed Generative-PLS model structure. Although the traditional PLS is a special case, this proposed GFLSRM structure leads to a potential inference structure for all the linear continuous latent variable methods.