Permutation-preserving Functions and Neural Vecchia Covariance Kernels

TL;DR

This paper presents a new neural network-based framework for learning scalable, non-stationary covariance kernels for Gaussian processes, leveraging permutation-equivariant structures for improved stability and efficiency.

Contribution

It introduces a universal neural architecture for permutation-preserving functions to learn covariance kernels directly from data, combining classical GP methods with deep learning.

Findings

Achieves stable and data-efficient training of covariance kernels.

Enables expressive, non-stationary kernel learning at scale.

Bridges classical Gaussian process methods with modern deep neural networks.

Abstract

We introduce a novel framework for constructing scalable and flexible covariance kernels for Gaussian processes (GPs) by directly learning the covariance structure under a regression-type parameterization induced by Vecchia approximations, using deep neural architectures. Specifically, we model kriging coefficients and conditional standard deviations, deterministic quantities that uniquely characterize the covariance, providing stable and informative learning targets. Exploiting the permutation-equivariant structure of conditioning sets in the Vecchia factorization, we derive a universal representation for permutation-preserving functions and design neural architectures that respect this symmetry, leading to improved training stability and data efficiency. The proposed approach enables expressive, non-stationary kernel learning while maintaining computational scalability, thereby bridging classical GP methodology with modern deep learning.