PAC-Bayes meta-learning with implicit task-specific posteriors

TL;DR

This paper presents a new PAC-Bayes meta-learning algorithm that effectively handles few-shot learning by extending the framework to multiple tasks and using a generative approach for task-specific posteriors, achieving state-of-the-art results.

Contribution

It introduces a PAC-Bayes meta-learning method with a generative posterior estimation, extending the framework to multiple tasks for improved few-shot learning performance.

Findings

Achieves state-of-the-art calibration and classification accuracy on few-shot benchmarks.

Demonstrates effective generalization to unseen tasks.

Provides a rigorous theoretical upper bound on task error.

Abstract

We introduce a new and rigorously-formulated PAC-Bayes meta-learning algorithm that solves few-shot learning. Our proposed method extends the PAC-Bayes framework from a single task setting to the meta-learning multiple task setting to upper-bound the error evaluated on any, even unseen, tasks and samples. We also propose a generative-based approach to estimate the posterior of task-specific model parameters more expressively compared to the usual assumption based on a multivariate normal distribution with a diagonal covariance matrix. We show that the models trained with our proposed meta-learning algorithm are well calibrated and accurate, with state-of-the-art calibration and classification results on few-shot classification (mini-ImageNet and tiered-ImageNet) and regression (multi-modal task-distribution regression) benchmarks.