DeGAS: Gradient-Based Optimization of Probabilistic Programs without Sampling

TL;DR

DeGAS introduces a differentiable, sample-free approach for optimizing probabilistic programs with continuous and discrete elements, enabling efficient gradient-based inference without Monte Carlo methods.

Contribution

It provides a novel Gaussian-mixture semantics for probabilistic programs that allows end-to-end gradient optimization without sampling, handling complex conditioning scenarios.

Findings

Achieves accuracy and speed comparable to variational inference and MCMC.

Successfully handles optimization problems where sampling methods struggle to converge.

Offers a differentiable, closed-form solution for probabilistic program inference.

Abstract

We present DeGAS, a differentiable Gaussian approximate semantics for loopless probabilistic programs that enables sample-free, gradient-based optimization in models with both continuous and discrete components. DeGAS evaluates programs under a Gaussian-mixture semantics and replaces measure-zero predicates and discrete branches with a vanishing smoothing, yielding closed-form expressions for posterior and path probabilities. We prove differentiability of these quantities with respect to program parameters, enabling end-to-end optimization via standard automatic differentiation, without Monte Carlo estimators. On thirteen benchmark programs, DeGAS achieves accuracy and runtime competitive with variational inference and MCMC. Importantly, it reliably tackles optimization problems where sampling-based baselines fail to converge due to conditioning involving continuous variables.