Single MCMC Chain Parallelisation on Decision Trees

TL;DR

This paper introduces a method to parallelize a single MCMC decision tree chain, significantly reducing computation time while maintaining statistical accuracy, making Bayesian decision trees more practical for high-dimensional data.

Contribution

The paper presents a novel approach to parallelize a single MCMC decision tree chain, enabling faster computation on multi-core systems without losing statistical validity.

Findings

Achieved up to 18x faster runtime with parallelization.

Parallel implementation produces statistically identical results to sequential.

Theoretical and practical runtime reductions are quantified.

Abstract

Decision trees are highly famous in machine learning and usually acquire state-of-the-art performance. Despite that, well-known variants like CART, ID3, random forest, and boosted trees miss a probabilistic version that encodes prior assumptions about tree structures and shares statistical strength between node parameters. Existing work on Bayesian decision trees depend on Markov Chain Monte Carlo (MCMC), which can be computationally slow, especially on high dimensional data and expensive proposals. In this study, we propose a method to parallelise a single MCMC decision tree chain on an average laptop or personal computer that enables us to reduce its run-time through multi-core processing while the results are statistically identical to conventional sequential implementation. We also calculate the theoretical and practical reduction in run time, which can be obtained utilising our method on multi-processor architectures. Experiments showed that we could achieve 18 times faster running time provided that the serial and the parallel implementation are statistically identical.