A Domain-Theoretic Approach to Statistical Programming Languages

TL;DR

This paper develops a domain-theoretic semantics for a statistical programming language using minimal valuations, enabling program transformations and exact real arithmetic, with demonstrations of expressive power and adequacy of semantics.

Contribution

It introduces a novel domain-theoretic semantics with a commutative monad of minimal valuations, bridging to implementations and showing expressive completeness for measures on the real line.

Findings

The monad of minimal valuations is commutative, allowing permutation of independent random draws.

All measures on the real line can be modeled by minimal valuations on the domain of exact real arithmetic.

Three operational semantics are shown to be adequate with respect to the denotational semantics.

Abstract

We give a domain-theoretic semantics to a statistical programming language, using the plain old category of dcpos, in contrast to some more sophisticated recent proposals. Remarkably, our monad of minimal valuations is commutative, which allows for program transformations that permute the order of independent random draws, as one would expect. A similar property is not known for Jones and Plotkin' s monad of continuous valuations. Instead of working with true real numbers, we work with exact real arithmetic, providing a bridge towards possible implementations. (Implementations by themselves are not addressed here.) Rather remarkably, we show that restricting ourselves to minimal valuations does not restrict us much: all measures on the real line can be modeled by minimal valuations on the domain $\mathbf{I}\mathbb{R}_\bot$ of exact real arithmetic. We give three operational semantics for our language, and we show that they are all adequate with respect to the denotational semantics. We also explore quite a few examples in order to demonstrate that our semantics computes exactly as one would expect, and in order to debunk the myth that a semantics based on continuous maps would not be expressive enough to encode measures with non-compact support using only measures with compact support, or to encode measures via non-continuous density functions, for instance. Our examples also include some useful, non-trivial cases of distributions on higher-order objects.