Accelerating Gradient-based Meta Learner

TL;DR

This paper proposes acceleration techniques for gradient-based meta learning algorithms like MAML, achieving significant speedups and improved accuracy by training tasks in clusters, thus reducing training time and enhancing performance.

Contribution

It introduces novel acceleration methods for meta learning, including task clustering, leading to faster convergence and better accuracy compared to existing approaches.

Findings

3.73X speedup on RNN optimizer-based meta learner

Task clustering improves training efficiency and accuracy

Significant reduction in training iterations required

Abstract

Meta Learning has been in focus in recent years due to the meta-learner model's ability to adapt well and generalize to new tasks, thus, reducing both the time and data requirements for learning. However, a major drawback of meta learner is that, to reach to a state from where learning new tasks becomes feasible with less data, it requires a large number of iterations and a lot of time. We address this issue by proposing various acceleration techniques to speed up meta learning algorithms such as MAML (Model Agnostic Meta Learning). We present 3.73X acceleration on a well known RNN optimizer based meta learner proposed in literature [11]. We introduce a novel method of training tasks in clusters, which not only accelerates the meta learning process but also improves model accuracy performance. Keywords: Meta learning, RNN optimizer, AGI, Performance optimization