Comparison of Gaussian process modeling software

TL;DR

This paper compares eight Gaussian process modeling software packages across different platforms, highlighting significant differences in results, prediction accuracy, and predictive variance, which impact model reliability and application.

Contribution

It provides a comprehensive comparison of popular Gaussian process software, revealing discrepancies and guiding users in selecting appropriate tools.

Findings

Significant differences in prediction results across packages.

Variations in predictive variance affect model confidence.

Performance differences depend on data functions and datasets.

Abstract

Gaussian process fitting, or kriging, is often used to create a model from a set of data. Many available software packages do this, but we show that very different results can be obtained from different packages even when using the same data and model. We describe the parameterization, features, and optimization used by eight different fitting packages that run on four different platforms. We then compare these eight packages using various data functions and data sets, revealing that there are stark differences between the packages. In addition to comparing the prediction accuracy, the predictive variance--which is important for evaluating precision of predictions and is often used in stopping criteria--is also evaluated.