Conditional Generative Moment-Matching Networks

TL;DR

This paper introduces Conditional Generative Moment-Matching Networks (CGMMN), a new deep generative model that learns conditional distributions using a kernel-based discrepancy measure, demonstrating competitive results across various tasks.

Contribution

The paper proposes CGMMN, a novel model that extends MMD-based generative networks to learn conditional distributions with a new CMMD criterion and back-propagation training.

Findings

CGMMN performs well on predictive modeling tasks.

It effectively handles contextual generation.

It successfully distills Bayesian knowledge into smaller networks.

Abstract

Maximum mean discrepancy (MMD) has been successfully applied to learn deep generative models for characterizing a joint distribution of variables via kernel mean embedding. In this paper, we present conditional generative moment- matching networks (CGMMN), which learn a conditional distribution given some input variables based on a conditional maximum mean discrepancy (CMMD) criterion. The learning is performed by stochastic gradient descent with the gradient calculated by back-propagation. We evaluate CGMMN on a wide range of tasks, including predictive modeling, contextual generation, and Bayesian dark knowledge, which distills knowledge from a Bayesian model by learning a relatively small CGMMN student network. Our results demonstrate competitive performance in all the tasks.