Rethinking supervised learning: insights from biological learning and from calling it by its name

TL;DR

This paper critically examines the role of supervision in learning, arguing that natural learning involves supervision and that rebranding or abandoning supervised learning may hinder progress in AI development.

Contribution

It challenges the notion that supervised learning is flawed by highlighting insights from biological learning and advocates for recognizing supervision as fundamental.

Findings

Supervised learning is central to effective generalisation.

Natural learning involves supervision and inductive biases.

Rebranding supervised learning may hinder scientific progress.

Abstract

The renaissance of artificial neural networks was catalysed by the success of classification models, tagged by the community with the broader term supervised learning. The extraordinary results gave rise to a hype loaded with ambitious promises and overstatements. Soon the community realised that the success owed much to the availability of thousands of labelled examples and supervised learning went, for many, from glory to shame: Some criticised deep learning as a whole and others proclaimed that the way forward had to be alternatives to supervised learning: predictive, unsupervised, semi-supervised and, more recently, self-supervised learning. However, all these seem brand names, rather than actual categories of a theoretically grounded taxonomy. Moreover, the call to banish supervised learning was motivated by the questionable claim that humans learn with little or no supervision and are capable of robust out-of-distribution generalisation. Here, we review insights about learning and supervision in nature, revisit the notion that learning and generalisation are not possible without supervision or inductive biases and argue that we will make better progress if we just call it by its name.