A Theory of Machine Learning

TL;DR

This paper reviews existing machine learning theories and introduces a new perspective where machines learn functions by successfully computing them, challenging traditional assumptions about probability and convergence.

Contribution

It proposes a novel theory of machine learning based on successful computation, contrasting with existing statistical and computational theories.

Findings

Challenges the assumption that learning true probabilities requires convergence

Shows that successful computation is sufficient for learning a function

Includes case studies in NLP and macroeconomics supporting the new theory

Abstract

We critically review three major theories of machine learning and provide a new theory according to which machines learn a function when the machines successfully compute it. We show that this theory challenges common assumptions in the statistical and the computational learning theories, for it implies that learning true probabilities is equivalent neither to obtaining a correct calculation of the true probabilities nor to obtaining an almost-sure convergence to them. We also briefly discuss some case studies from natural language processing and macroeconomics from the perspective of the new theory.