A Markov Decision Process Approach to Active Meta Learning

TL;DR

This paper introduces an active meta-learning method using Markov Decision Processes to select training samples efficiently, significantly reducing sample complexity by exploiting task relationships.

Contribution

It formulates sample selection in meta-learning as an MDP and develops scheduling schemes based on UCB, Gittins Index, and linear programming, which is a novel approach.

Findings

Significant reduction in sample complexity compared to random sampling.

Effective exploitation of covariates improves meta-learning performance.

Versatile application across various experimental contexts.

Abstract

In supervised learning, we fit a single statistical model to a given data set, assuming that the data is associated with a singular task, which yields well-tuned models for specific use, but does not adapt well to new contexts. By contrast, in meta-learning, the data is associated with numerous tasks, and we seek a model that may perform well on all tasks simultaneously, in pursuit of greater generalization. One challenge in meta-learning is how to exploit relationships between tasks and classes, which is overlooked by commonly used random or cyclic passes through data. In this work, we propose actively selecting samples on which to train by discerning covariates inside and between meta-training sets. Specifically, we cast the problem of selecting a sample from a number of meta-training sets as either a multi-armed bandit or a Markov Decision Process (MDP), depending on how one encapsulates correlation across tasks. We develop scheduling schemes based on Upper Confidence Bound (UCB), Gittins Index and tabular Markov Decision Problems (MDPs) solved with linear programming, where the reward is the scaled statistical accuracy to ensure it is a time-invariant function of state and action. Across a variety of experimental contexts, we observe significant reductions in sample complexity of active selection scheme relative to cyclic or i.i.d. sampling, demonstrating the merit of exploiting covariates in practice.