Hypernetwork approach to Bayesian MAML

TL;DR

This paper introduces BayesianHMAML, a hypernetwork-based Bayesian extension of MAML, which improves uncertainty quantification and reduces overfitting in few-shot learning by using flexible posterior distributions.

Contribution

It proposes a novel hypernetwork approach for Bayesian MAML that incorporates complex posterior distributions, enhancing uncertainty modeling and adaptability.

Findings

Enables learning of complex posterior distributions with hypernetworks.

Reduces overfitting in few-shot learning scenarios.

Improves uncertainty quantification over traditional Gaussian-based methods.

Abstract

The main goal of Few-Shot learning algorithms is to enable learning from small amounts of data. One of the most popular and elegant Few-Shot learning approaches is Model-Agnostic Meta-Learning (MAML). The main idea behind this method is to learn the shared universal weights of a meta-model, which are then adapted for specific tasks. However, the method suffers from over-fitting and poorly quantifies uncertainty due to limited data size. Bayesian approaches could, in principle, alleviate these shortcomings by learning weight distributions in place of point-wise weights. Unfortunately, previous modifications of MAML are limited due to the simplicity of Gaussian posteriors, MAML-like gradient-based weight updates, or by the same structure enforced for universal and adapted weights. In this paper, we propose a novel framework for Bayesian MAML called BayesianHMAML, which employs Hypernetworks for weight updates. It learns the universal weights point-wise, but a probabilistic structure is added when adapted for specific tasks. In such a framework, we can use simple Gaussian distributions or more complicated posteriors induced by Continuous Normalizing Flows.