Adaptive Gradient-Based Meta-Learning Methods

TL;DR

This paper develops a theoretical framework for adaptive gradient-based meta-learning that learns task similarity, improves transfer-risk bounds, and enhances performance in few-shot and federated learning scenarios.

Contribution

It introduces a unified theory for meta-learning that incorporates task similarity, leading to sharper bounds and improved algorithms for dynamic and structured task environments.

Findings

Sharper transfer-risk bounds in statistical learning-to-learn.

Improved meta-test-time performance in few-shot learning.

Enhanced algorithms for federated learning.

Abstract

We build a theoretical framework for designing and understanding practical meta-learning methods that integrates sophisticated formalizations of task-similarity with the extensive literature on online convex optimization and sequential prediction algorithms. Our approach enables the task-similarity to be learned adaptively, provides sharper transfer-risk bounds in the setting of statistical learning-to-learn, and leads to straightforward derivations of average-case regret bounds for efficient algorithms in settings where the task-environment changes dynamically or the tasks share a certain geometric structure. We use our theory to modify several popular meta-learning algorithms and improve their meta-test-time performance on standard problems in few-shot learning and federated learning.