Convergence of Contrastive Divergence with Annealed Learning Rate in Exponential Family

TL;DR

This paper proves the consistency and convergence rate of contrastive divergence with annealed learning rate in exponential family models, extending previous fixed-rate results and providing theoretical guarantees.

Contribution

It establishes the convergence properties of CD with annealed learning rate, including the rate of convergence and the impact of MCMC steps, using a novel super-martingale approach.

Findings

Convergence rate of 1/n^{1/3} for parameter estimates.

Number of MCMC steps affects convergence coefficient.

Experimental validation on a Boltzmann Machine.

Abstract

In our recent paper, we showed that in exponential family, contrastive divergence (CD) with fixed learning rate will give asymptotically consistent estimates \cite{wu2016convergence}. In this paper, we establish consistency and convergence rate of CD with annealed learning rate $\eta_t$. Specifically, suppose CD-$m$ generates the sequence of parameters $\{\theta_t\}_{t \ge 0}$ using an i.i.d. data sample $\mathbf{X}_1^n \sim p_{\theta^*}$ of size $n$, then $\delta_n(\mathbf{X}_1^n) = \limsup_{t \to \infty} \Vert \sum_{s=t_0}^t \eta_s \theta_s / \sum_{s=t_0}^t \eta_s - \theta^* \Vert$ converges in probability to 0 at a rate of $1/\sqrt[3]{n}$. The number ($m$) of MCMC transitions in CD only affects the coefficient factor of convergence rate. Our proof is not a simple extension of the one in \cite{wu2016convergence}. which depends critically on the fact that $\{\theta_t\}_{t \ge 0}$ is a homogeneous Markov chain conditional on the observed sample $\mathbf{X}_1^n$. Under annealed learning rate, the homogeneous Markov property is not available and we have to develop an alternative approach based on super-martingales. Experiment results of CD on a fully-visible $2\times 2$ Boltzmann Machine are provided to demonstrate our theoretical results.