Kernels, Data & Physics

TL;DR

This paper provides lecture notes on the NTK approach in machine learning, emphasizing practical applications like data distillation and adversarial robustness, and discusses inductive biases in kernel methods.

Contribution

It offers an accessible overview of the NTK framework with a focus on practical problems and insights into inductive biases in kernel-based learning.

Findings

NTK approach aids in understanding complex ML problems

Kernel methods improve data distillation techniques

Insights into adversarial robustness through kernels

Abstract

Lecture notes from the course given by Professor Julia Kempe at the summer school "Statistical physics of Machine Learning" in Les Houches. The notes discuss the so-called NTK approach to problems in machine learning, which consists of gaining an understanding of generally unsolvable problems by finding a tractable kernel formulation. The notes are mainly focused on practical applications such as data distillation and adversarial robustness, examples of inductive bias are also discussed.