Noise Contrastive Meta-Learning for Conditional Density Estimation using Kernel Mean Embeddings

TL;DR

This paper introduces a novel meta-learning method for conditional density estimation that combines neural representations, noise-contrastive estimation, and kernel mean embeddings, effectively handling multimodal distributions.

Contribution

It presents a new approach integrating neural networks and kernel mean embeddings for meta-learning conditional densities, addressing multimodality beyond expectations.

Findings

Effective on synthetic and real-world datasets

Shares learned representations across multiple tasks

Handles multimodal conditional distributions

Abstract

Current meta-learning approaches focus on learning functional representations of relationships between variables, i.e. on estimating conditional expectations in regression. In many applications, however, we are faced with conditional distributions which cannot be meaningfully summarized using expectation only (due to e.g. multimodality). Hence, we consider the problem of conditional density estimation in the meta-learning setting. We introduce a novel technique for meta-learning which combines neural representation and noise-contrastive estimation with the established literature of conditional mean embeddings into reproducing kernel Hilbert spaces. The method is validated on synthetic and real-world problems, demonstrating the utility of sharing learned representations across multiple conditional density estimation tasks.