Joint-stochastic-approximation Random Fields with Application to Semi-supervised Learning

TL;DR

This paper introduces joint-stochastic-approximation random fields (JRFs), a new method for deep undirected generative models that effectively balances mode coverage and missing modes, improving semi-supervised learning performance.

Contribution

The paper proposes JRFs, a novel algorithmic framework for deep undirected generative models, addressing mode collapse and classification-generation conflicts in SSL.

Findings

JRFs balance mode covering and missing modes effectively.

JRFs achieve competitive classification accuracy on MNIST, SVHN, CIFAR-10.

JRFs perform well in both generation quality and classification tasks.

Abstract

Our examination of deep generative models (DGMs) developed for semi-supervised learning (SSL), mainly GANs and VAEs, reveals two problems. First, mode missing and mode covering phenomenons are observed in genertion with GANs and VAEs. Second, there exists an awkward conflict between good classification and good generation in SSL by employing directed generative models. To address these problems, we formally present joint-stochastic-approximation random fields (JRFs) -- a new family of algorithms for building deep undirected generative models, with application to SSL. It is found through synthetic experiments that JRFs work well in balancing mode covering and mode missing, and match the empirical data distribution well. Empirically, JRFs achieve good classification results comparable to the state-of-art methods on widely adopted datasets -- MNIST, SVHN, and CIFAR-10 in SSL, and simultaneously perform good generation.