Meta-Learning of Structured Task Distributions in Humans and Machines

TL;DR

This paper compares human and machine meta-learning on structured tasks generated by a grammar, revealing humans excel with structure while agents perform better on unstructured, statistically similar tasks, highlighting differences in learning strategies.

Contribution

It introduces a new structured task distribution and a null control distribution, and compares human and machine meta-learning, revealing fundamental differences in their use of task structure.

Findings

Humans outperform machines on structured tasks.

Machines perform better on unstructured, null tasks.

Meta-learners may not utilize task structure as humans do.

Abstract

In recent years, meta-learning, in which a model is trained on a family of tasks (i.e. a task distribution), has emerged as an approach to training neural networks to perform tasks that were previously assumed to require structured representations, making strides toward closing the gap between humans and machines. However, we argue that evaluating meta-learning remains a challenge, and can miss whether meta-learning actually uses the structure embedded within the tasks. These meta-learners might therefore still be significantly different from humans learners. To demonstrate this difference, we first define a new meta-reinforcement learning task in which a structured task distribution is generated using a compositional grammar. We then introduce a novel approach to constructing a "null task distribution" with the same statistical complexity as this structured task distribution but without the explicit rule-based structure used to generate the structured task. We train a standard meta-learning agent, a recurrent network trained with model-free reinforcement learning, and compare it with human performance across the two task distributions. We find a double dissociation in which humans do better in the structured task distribution whereas agents do better in the null task distribution -- despite comparable statistical complexity. This work highlights that multiple strategies can achieve reasonable meta-test performance, and that careful construction of control task distributions is a valuable way to understand which strategies meta-learners acquire, and how they might differ from humans.