Understanding Kernel Ridge Regression: Common behaviors from simple functions to density functionals

TL;DR

This paper investigates the behavior of Kernel Ridge Regression in modeling simple functions and density functionals, revealing universal error features across different regimes and hyperparameters, enhancing understanding of ML approximation limits.

Contribution

It provides a detailed analysis of Kernel Ridge Regression's error behavior in simple and density functional models, highlighting universal features and limiting behaviors.

Findings

Universal error features in extreme limits

Behavioral insights across hyperparameters

Error dependence on length scales and noise

Abstract

Accurate approximations to density functionals have recently been obtained via machine learning (ML). By applying ML to a simple function of one variable without any random sampling, we extract the qualitative dependence of errors on hyperparameters. We find universal features of the behavior in extreme limits, including both very small and very large length scales, and the noise-free limit. We show how such features arise in ML models of density functionals.