Transfer Bayesian Meta-learning via Weighted Free Energy Minimization

TL;DR

This paper introduces weighted free energy minimization (WFEM) for transfer meta-learning to handle distribution shifts between training and testing tasks, validated on regression and classification benchmarks.

Contribution

It proposes a novel WFEM approach for transfer meta-learning, specifically addressing distribution mismatch issues in Bayesian meta-learning with Gaussian Processes.

Findings

WFEM effectively manages task distribution shifts in meta-learning.

The method outperforms standard GP-based meta-learning on benchmark datasets.

Validation on toy and real-world datasets demonstrates improved adaptability.

Abstract

Meta-learning optimizes the hyperparameters of a training procedure, such as its initialization, kernel, or learning rate, based on data sampled from a number of auxiliary tasks. A key underlying assumption is that the auxiliary tasks, known as meta-training tasks, share the same generating distribution as the tasks to be encountered at deployment time, known as meta-test tasks. This may, however, not be the case when the test environment differ from the meta-training conditions. To address shifts in task generating distribution between meta-training and meta-testing phases, this paper introduces weighted free energy minimization (WFEM) for transfer meta-learning. We instantiate the proposed approach for non-parametric Bayesian regression and classification via Gaussian Processes (GPs). The method is validated on a toy sinusoidal regression problem, as well as on classification using miniImagenet and CUB data sets, through comparison with standard meta-learning of GP priors as implemented by PACOH.