Random Sampling High Dimensional Model Representation Gaussian Process Regression (RS-HDMR-GPR) for representing multidimensional functions with machine-learned lower-dimensional terms allowing insight with a general method

TL;DR

This paper introduces RS-HDMR-GPR, a Python-based method combining high-dimensional model representation with Gaussian process regression to efficiently model complex multivariate functions, especially from sparse data, with added interpretability.

Contribution

The paper presents a novel Python implementation of RS-HDMR-GPR that enables low-dimensional representation, missing data imputation, and variable importance estimation for multidimensional functions.

Findings

Effective in recovering functional dependence from sparse data

Capable of imputing missing variable values

Provides insights into variable importance

Abstract

We present a Python implementation for RS-HDMR-GPR (Random Sampling High Dimensional Model Representation Gaussian Process Regression). The method builds representations of multivariate functions with lower-dimensional terms, either as an expansion over orders of coupling or using terms of only a given dimensionality. This facilitates, in particular, recovering functional dependence from sparse data. The code also allows for imputation of missing values of the variables and for a significant pruning of the useful number of HDMR terms. The code can also be used for estimating relative importance of different combinations of input variables, thereby adding an element of insight to a general machine learning method. The capabilities of this regression tool are demonstrated on test cases involving synthetic analytic functions, the potential energy surface of the water molecule, kinetic energy densities of materials (crystalline magnesium, aluminum, and silicon), and financial market data.