Covariate Distribution Aware Meta-learning

TL;DR

This paper introduces a hierarchical Bayesian meta-learning approach that explicitly models the covariate distribution to improve task inference and performance across classification and regression tasks.

Contribution

It proposes a novel graphical model and meta-learning algorithm that incorporate covariate distribution information, addressing a gap in existing Bayesian meta-learning methods.

Findings

Improved accuracy on classification benchmarks.

Enhanced task inference in synthetic regression experiments.

Demonstrated benefits of covariate modeling over traditional methods.

Abstract

Meta-learning has proven to be successful for few-shot learning across the regression, classification, and reinforcement learning paradigms. Recent approaches have adopted Bayesian interpretations to improve gradient-based meta-learners by quantifying the uncertainty of the post-adaptation estimates. Most of these works almost completely ignore the latent relationship between the covariate distribution $(p(x))$ of a task and the corresponding conditional distribution $p(y|x)$. In this paper, we identify the need to explicitly model the meta-distribution over the task covariates in a hierarchical Bayesian framework. We begin by introducing a graphical model that leverages the samples from the marginal $p(x)$ to better infer the posterior over the optimal parameters of the conditional distribution $(p(y|x))$ for each task. Based on this model we propose a computationally feasible meta-learning algorithm by introducing meaningful relaxations in our final objective. We demonstrate the gains of our algorithm over initialization based meta-learning baselines on popular classification benchmarks. Finally, to understand the potential benefit of modeling task covariates we further evaluate our method on a synthetic regression dataset.