The adaptive complexity of parallelized log-concave sampling

TL;DR

This paper investigates the minimal number of sequential rounds needed for parallelized sampling algorithms to generate accurate samples from log-concave distributions, revealing fundamental limitations in their efficiency.

Contribution

It introduces new lower bounds on the adaptive complexity of parallelized sampling for log-concave distributions, highlighting inherent computational challenges.

Findings

Almost linear iteration algorithms cannot achieve exponentially small total variation error.

For box-constrained sampling, such algorithms cannot attain sup-polynomially small error.

The proofs involve novel analysis of hardness potentials and smoothing techniques.

Abstract

In large-data applications, such as the inference process of diffusion models, it is desirable to design sampling algorithms with a high degree of parallelization. In this work, we study the adaptive complexity of sampling, which is the minimum number of sequential rounds required to achieve sampling given polynomially many queries executed in parallel at each round. For unconstrained sampling, we examine distributions that are log-smooth or log-Lipschitz and log strongly or non-strongly concave. We show that an almost linear iteration algorithm cannot return a sample with a specific exponentially small error under total variation distance. For box-constrained sampling, we show that an almost linear iteration algorithm cannot return a sample with sup-polynomially small error under total variation distance for log-concave distributions. Our proof relies upon novel analysis with the characterization of the output for the hardness potentials based on the chain-like structure with random partition and classical smoothing techniques.