Population-Contrastive-Divergence: Does Consistency help with RBM training?

TL;DR

This paper introduces Population-Contrastive-Divergence (pop-CD), a new RBM training algorithm that offers a consistent gradient estimate with lower bias, improving training accuracy over traditional Contrastive Divergence, especially on smaller models.

Contribution

The paper proposes pop-CD, a novel RBM training method based on Population Monte Carlo, providing a consistent gradient estimate with minimal additional computational cost.

Findings

pop-CD reduces bias compared to CD

pop-CD achieves higher log-likelihood values

pop-CD's variance increases, affecting large models

Abstract

Estimating the log-likelihood gradient with respect to the parameters of a Restricted Boltzmann Machine (RBM) typically requires sampling using Markov Chain Monte Carlo (MCMC) techniques. To save computation time, the Markov chains are only run for a small number of steps, which leads to a biased estimate. This bias can cause RBM training algorithms such as Contrastive Divergence (CD) learning to deteriorate. We adopt the idea behind Population Monte Carlo (PMC) methods to devise a new RBM training algorithm termed Population-Contrastive-Divergence (pop-CD). Compared to CD, it leads to a consistent estimate and may have a significantly lower bias. Its computational overhead is negligible compared to CD. However, the variance of the gradient estimate increases. We experimentally show that pop-CD can significantly outperform CD. In many cases, we observed a smaller bias and achieved higher log-likelihood values. However, when the RBM distribution has many hidden neurons, the consistent estimate of pop-CD may still have a considerable bias and the variance of the gradient estimate requires a smaller learning rate. Thus, despite its superior theoretical properties, it is not advisable to use pop-CD in its current form on large problems.