Scalable Physics-based Maximum Likelihood Estimation using Hierarchical Matrices

TL;DR

This paper introduces a scalable, matrix-free hierarchical matrix approach for efficient maximum likelihood estimation in physics-based Gaussian process models, significantly reducing computational and storage costs.

Contribution

It develops a novel hierarchical matrix approximation method with randomized sketching for covariance functions, enabling efficient parameter estimation and uncertainty quantification.

Findings

Achieves $O(n ext{log}^2 n)$ complexity for covariance matrix construction and likelihood computation.

Provides an exact trace computation method for hierarchical matrices, facilitating Fisher information calculation.

Demonstrates high accuracy and efficiency in numerical experiments for parameter estimation.

Abstract

Physics-based covariance models provide a systematic way to construct covariance models that are consistent with the underlying physical laws in Gaussian process analysis. The unknown parameters in the covariance models can be estimated using maximum likelihood estimation, but direct construction of the covariance matrix and classical strategies of computing with it requires $n$ physical model runs, $n^2$ storage complexity, and $n^3$ computational complexity. To address such challenges, we propose to approximate the discretized covariance function using hierarchical matrices. By utilizing randomized range sketching for individual off-diagonal blocks, the construction process of the hierarchical covariance approximation requires $O(\log{n})$ physical model applications and the maximum likelihood computations require $O(n\log^2{n})$ effort per iteration. We propose a new approach to compute exactly the trace of products of hierarchical matrices which results in the expected Fischer information matrix being computable in $O(n\log^2{n})$ as well. The construction is totally matrix-free and the derivatives of the covariance matrix can then be approximated in the same hierarchical structure by differentiating the whole process. Numerical results are provided to demonstrate the effectiveness, accuracy, and efficiency of the proposed method for parameter estimations and uncertainty quantification.