Group kernels for Gaussian process metamodels with categorical inputs

TL;DR

This paper develops a new class of covariance functions for Gaussian process models with categorical inputs, especially when the number of categories is large, by leveraging block covariance matrices and hierarchical structures.

Contribution

It introduces a flexible parametric family of valid covariance matrices for categorical inputs, ensuring positive definiteness through a hierarchical block structure and averaging techniques.

Findings

The proposed covariance model is valid for large numbers of categories.

Application to nuclear waste analysis demonstrates practical effectiveness.

The hierarchical approach simplifies positive definiteness verification.

Abstract

Gaussian processes (GP) are widely used as a metamodel for emulating time-consuming computer codes. We focus on problems involving categorical inputs, with a potentially large number L of levels (typically several tens), partitioned in G << L groups of various sizes. Parsimonious covariance functions, or kernels, can then be defined by block covariance matrices T with constant covariances between pairs of blocks and within blocks. We study the positive definiteness of such matrices to encourage their practical use. The hierarchical group/level structure, equivalent to a nested Bayesian linear model, provides a parameterization of valid block matrices T. The same model can then be used when the assumption within blocks is relaxed, giving a flexible parametric family of valid covariance matrices with constant covariances between pairs of blocks. The positive definiteness of T is equivalent to the positive definiteness of a smaller matrix of size G, obtained by averaging each block. The model is applied to a problem in nuclear waste analysis, where one of the categorical inputs is atomic number, which has more than 90 levels.