Iterative conditional replacement algorithm for conditionally specified models

TL;DR

The paper introduces the Iterative Conditional Replacement (ICR) algorithm, a new method for approximating distributions in conditionally specified models that converges without Markov chains and improves over existing approaches.

Contribution

It presents ICR, a novel algorithm that converges to stationary distributions in conditionally specified models without relying on Markov chains, and introduces an ensemble method for model selection.

Findings

ICR always converges and produces mutually stationary distributions.

ICR outperforms existing methods in quality and parallelizability.

An ensemble approach effectively determines the final model.

Abstract

The sample-based Gibbs sampler has been the dominant method for approximating joint distribution from a collection of compatible full-conditional distributions. However for conditionally specified model, mixtures of incompatible full and non-full conditional distributions are the realities; but, their updating orders are hard to identified. We propose a new algorithm, the Iterative Conditional Replacement (ICR), that produces distributional approximations toward the stationary distributions, dispensing Markov chain entirely. ICR always converges, and it produces mutually stationary distributions, which will be consistent among one another when the conditional distributions are compatible. Examples show ICR to be superior in quality, while being more parallelizable and requiring little effort in monitoring its convergence. Last, we propose an ensemble approach to decide the final model.