A Theory of Human-Like Few-Shot Learning

TL;DR

This paper develops a theoretical framework for human-like few-shot learning based on physical principles, demonstrating that certain models can approximate this theory and outperform traditional neural networks in low-resource tasks.

Contribution

It introduces a new theory of human-like few-shot learning derived from physical principles and shows how generative models can effectively implement it.

Findings

Deep generative models outperform neural networks in low-resource tasks.

The theory aligns with existing models like Free Energy Principle and Bayesian Program Learning.

VAE can approximate the proposed human-like learning theory.

Abstract

We aim to bridge the gap between our common-sense few-sample human learning and large-data machine learning. We derive a theory of human-like few-shot learning from von-Neuman-Landauer's principle. modelling human learning is difficult as how people learn varies from one to another. Under commonly accepted definitions, we prove that all human or animal few-shot learning, and major models including Free Energy Principle and Bayesian Program Learning that model such learning, approximate our theory, under Church-Turing thesis. We find that deep generative model like variational autoencoder (VAE) can be used to approximate our theory and perform significantly better than baseline models including deep neural networks, for image recognition, low resource language processing, and character recognition.