Schoenberg characterization of continuous non-stationary isotropic positive definite kernels

TL;DR

This paper characterizes continuous, isotropic, positive definite kernels on Euclidean space, unifying stationary and neural network kernels, and provides conditions for strict positive definiteness.

Contribution

It offers a comprehensive characterization of invariant kernels on , including neural network kernels, and establishes criteria for strict positive definiteness.

Findings

Unified class of isotropic kernels including neural network kernels

Necessary and sufficient conditions for strict positive definiteness

General framework for continuous invariant kernels

Abstract

We provide a characterization for the continuous positive definite kernels on $\mathbb R^d$ that are invariant to linear isometries, i.e. invariant under the orthogonal group $O(d)$. Furthermore, we provide necessary and sufficient conditions for these kernels to be strictly positive definite. This class of isotropic kernels is fairly general: First, it unifies stationary isotropic and dot product kernels, and second, it includes neural network kernels that arise from infinite-width limits of neural networks.