Ridge Kernel Averaging and Uniform Approximation

TL;DR

This paper introduces a framework for function classes generated by parametric ridge kernels, characterizes their universality and expressivity, and analyzes the approximation and generalization properties of related random-kernel networks.

Contribution

It develops a comprehensive theory for ridge kernel classes, including universality criteria, expressivity dichotomy, and analysis of random-kernel networks with indefinite activations.

Findings

Ridge kernel classes are isometric to their RKHS.

A sharp universality criterion based on the conic hull of ridge atoms.

Monte Carlo rate of convergence for random-kernel networks with indefinite activations.

Abstract

We develop a framework for function classes generated by parametric ridge kernels: one-dimensional kernels composed with affine projections and averaged over a parameter measure. The induced kernels are positive definite, and the resulting integral class coincides isometrically with its reproducing kernel Hilbert space. We characterize all kernels obtainable by varying the measure as the uniform closure of the conic hull of ridge atoms, giving a sharp universality criterion; a slice-wise polynomial versus non-polynomial dichotomy governs expressivity. We then analyze random-kernel networks whose activations may be indefinite but have a positive-definite mean. For these networks we prove a Monte Carlo rate with mean-squared error of order one over $N$ and a high-probability uniform bound on compact sets, without requiring pathwise positive definiteness.