The Teaching Dimension of Kernel Perceptron

TL;DR

This paper investigates the teaching dimension of kernel perceptrons, extending the understanding of sample complexity from linear to nonlinear models, and provides bounds and examples for various kernels.

Contribution

It establishes the teaching complexity for kernel perceptrons with different kernels and data assumptions, advancing the theoretical understanding of machine teaching for nonlinear learners.

Findings

Teaching complexity is Θ(d) for linear perceptrons in ℝ^d.

Teaching complexity is Θ(d^k) for polynomial kernel perceptrons of order k.

Bounded the complexity for Gaussian kernel perceptrons under certain data assumptions.

Abstract

Algorithmic machine teaching has been studied under the linear setting where exact teaching is possible. However, little is known for teaching nonlinear learners. Here, we establish the sample complexity of teaching, aka teaching dimension, for kernelized perceptrons for different families of feature maps. As a warm-up, we show that the teaching complexity is $\Theta(d)$ for the exact teaching of linear perceptrons in $\mathbb{R}^d$, and $\Theta(d^k)$ for kernel perceptron with a polynomial kernel of order $k$. Furthermore, under certain smooth assumptions on the data distribution, we establish a rigorous bound on the complexity for approximately teaching a Gaussian kernel perceptron. We provide numerical examples of the optimal (approximate) teaching set under several canonical settings for linear, polynomial and Gaussian kernel perceptrons.