Gaussian Process Models with Low-Rank Correlation Matrices for Both Continuous and Categorical Inputs

TL;DR

This paper introduces Low-Rank Correlation (LRC), a flexible Gaussian Process modeling approach that efficiently handles mixed continuous and categorical inputs by approximating cross-correlation matrices with adjustable rank.

Contribution

The paper proposes the LRC method, enabling adaptable low-rank approximations of cross-correlation matrices in Gaussian Processes for mixed input types, improving estimation and prediction accuracy.

Findings

LRC performs well in estimating cross-correlations.

LRC improves response surface prediction accuracy.

LRC is especially effective with many categorical level combinations.

Abstract

We introduce a method that uses low-rank approximations of cross-correlation matrices in mixed continuous and categorical Gaussian Process models. This new method -- called Low-Rank Correlation (LRC) -- offers the ability to flexibly adapt the number of parameters to the problem at hand by choosing an appropriate rank of the approximation. Furthermore, we present a systematic approach of defining test functions that can be used for assessing the accuracy of models or optimization methods that are concerned with both continuous and categorical inputs. We compare LRC to existing approaches of modeling the cross-correlation matrix. It turns out that the new approach performs well in terms of estimation of cross-correlations and response surface prediction. Therefore, LRC is a flexible and useful addition to existing methods, especially for increasing numbers of combinations of levels of the categorical inputs.