The Human Kernel

TL;DR

This paper develops a kernel learning framework to understand and replicate human inductive biases in function extrapolation, providing psychological insights and improving Gaussian process models beyond traditional kernels.

Contribution

It introduces a method to reverse engineer human inductive biases through learned kernels and applies this to enhance function extrapolation in Gaussian processes.

Findings

Learned kernels capture human-like extrapolation behavior

Humans exhibit biases that differ from traditional kernels

Incorporating human biases improves model extrapolation

Abstract

Bayesian nonparametric models, such as Gaussian processes, provide a compelling framework for automatic statistical modelling: these models have a high degree of flexibility, and automatically calibrated complexity. However, automating human expertise remains elusive; for example, Gaussian processes with standard kernels struggle on function extrapolation problems that are trivial for human learners. In this paper, we create function extrapolation problems and acquire human responses, and then design a kernel learning framework to reverse engineer the inductive biases of human learners across a set of behavioral experiments. We use the learned kernels to gain psychological insights and to extrapolate in human-like ways that go beyond traditional stationary and polynomial kernels. Finally, we investigate Occam's razor in human and Gaussian process based function learning.