A General framework for PAC-Bayes Bounds for Meta-Learning

TL;DR

This paper introduces new PAC-Bayes bounds for meta-learning that account for environment and task-level generalization gaps, leading to improved algorithms with demonstrated advantages over previous bounds.

Contribution

It develops a general framework for PAC-Bayes bounds in meta-learning, incorporating both environment and task-level gaps, and proposes new algorithms based on these bounds.

Findings

New PAC-Bayes bounds for meta-learning that include environment and task-level gaps

Development of novel PAC-Bayes meta-learning algorithms

Numerical results show improved performance over prior bounds

Abstract

Meta learning automatically infers an inductive bias, that includes the hyperparameter of the base-learning algorithm, by observing data from a finite number of related tasks. This paper studies PAC-Bayes bounds on meta generalization gap. The meta-generalization gap comprises two sources of generalization gaps: the environment-level and task-level gaps resulting from observation of a finite number of tasks and data samples per task, respectively. In this paper, by upper bounding arbitrary convex functions, which link the expected and empirical losses at the environment and also per-task levels, we obtain new PAC-Bayes bounds. Using these bounds, we develop new PAC-Bayes meta-learning algorithms. Numerical examples demonstrate the merits of the proposed novel bounds and algorithm in comparison to prior PAC-Bayes bounds for meta-learning.