Composite Inference for Gaussian Processes

TL;DR

This paper introduces a composite inference method for large-scale Gaussian processes that efficiently estimates parameters and makes predictions by dividing data into subsets and leveraging conditional densities, overcoming computational challenges.

Contribution

It develops a novel composite likelihood approach that enables tractable parameter estimation and prediction for large Gaussian process models by exploiting data partitioning and dependence structures.

Findings

The method transforms intractable BLUP into a convex optimization problem.

The proposed predictor achieves minimum variance among block-based predictions.

It demonstrates computational efficiency and statistical accuracy in large datasets.

Abstract

Large-scale Gaussian process models are becoming increasingly important and widely used in many areas, such as, computer experiments, stochastic optimization via simulation, and machine learning using Gaussian processes. The standard methods, such as maximum likelihood estimation (MLE) for parameter estimation and the best linear unbiased predictor (BLUP) for prediction, are generally the primary choices in many applications. In spite of their merits, those methods are not feasible due to intractable computation when the sample size is huge. A novel method for the purposes of parameter estimation and prediction is proposed to solve the computational problems of large-scale Gaussian process based models, by separating the original dataset into tractable subsets. This method consistently combines parameter estimation and prediction by making full use of the dependence among conditional densities: a statistically efficient composite likelihood based on joint distributions of some well selected conditional densities is developed to estimate parameters and then "composite inference" is coined to make prediction for an unknown input point, based on its distributions conditional on each block subset. The proposed method transforms the intractable BLUP into a tractable convex optimization problem. It is also shown that the prediction given by the proposed method, called the best linear unbiased block predictor, has a minimum variance for a given separation of the dataset. Keywords: Large scale, Parallel computing, Composite likelihood, Spatial process