Parallel Simulation for Log-concave Sampling and Score-based Diffusion Models

TL;DR

This paper introduces a new parallel sampling method for high-dimensional log-concave distributions that significantly reduces adaptive complexity, achieving near-optimal efficiency and outperforming previous techniques.

Contribution

We propose a novel parallel sampling algorithm that improves adaptive complexity dependence on dimension from ( ext{log}^2 d) to ( ext{log} d), approaching optimality for log-concave sampling.

Findings

Reduces adaptive complexity dependence from ( ext{log}^2 d) to ( ext{log} d)

Achieves near-optimal parallel sampling efficiency for high-dimensional log-concave distributions

Builds on scientific computing parallel simulation techniques

Abstract

Sampling from high-dimensional probability distributions is fundamental in machine learning and statistics. As datasets grow larger, computational efficiency becomes increasingly important, particularly in reducing adaptive complexity, namely the number of sequential rounds required for sampling algorithms. While recent works have introduced several parallelizable techniques, they often exhibit suboptimal convergence rates and remain significantly weaker than the latest lower bounds for log-concave sampling. To address this, we propose a novel parallel sampling method that improves adaptive complexity dependence on dimension $d$ reducing it from $\widetilde{\mathcal{O}}(\log^2 d)$ to $\widetilde{\mathcal{O}}(\log d)$. which is even optimal for log-concave sampling with some specific adaptive complexity. Our approach builds on parallel simulation techniques from scientific computing.