SIMPL: A DSL for Automatic Specialization of Inference Algorithms

TL;DR

SIMPL is a domain-specific language that automatically specializes inference algorithms in probabilistic programming, significantly improving their efficiency through partial evaluation, caching, and C code generation.

Contribution

Introduces SIMPL, a DSL for automatic inference algorithm specialization, enabling easy addition and optimization of algorithms using domain-specific information.

Findings

Partial evaluation yields 2-6x speedup.

Caching provides an additional 1-1.5x speedup.

C code generation results in 13-20x speedup.

Abstract

Inference algorithms in probabilistic programming languages (PPLs) can be thought of as interpreters, since an inference algorithm traverses a model given evidence to answer a query. As with interpreters, we can improve the efficiency of inference algorithms by compiling them once the model, evidence and query are known. We present SIMPL, a domain specific language for inference algorithms, which uses this idea in order to automatically specialize annotated inference algorithms. Due to the approach of specialization, unlike a traditional compiler, with SIMPL new inference algorithms can be added easily, and still be optimized using domain-specific information. We evaluate SIMPL and show that partial evaluation gives a 2-6x speedup, caching provides an additional 1-1.5x speedup, and generating C code yields an additional 13-20x speedup, for an overall speedup of 30-150x for several inference algorithms and models.