P(Expression|Grammar): Probability of deriving an algebraic expression with a probabilistic context-free grammar

TL;DR

This paper investigates the probability of deriving algebraic expressions from probabilistic context-free grammars, proving undecidability in general and providing algorithms for specific classes like linear and polynomial expressions.

Contribution

It introduces a formal framework for computing expression probabilities and develops algorithms for exact and approximate calculations for certain grammars.

Findings

Decidability of expression probability is limited in general cases.

Algorithms for exact probability calculation in specific grammars.

Efficient approximation methods with arbitrary precision.

Abstract

Probabilistic context-free grammars have a long-term record of use as generative models in machine learning and symbolic regression. When used for symbolic regression, they generate algebraic expressions. We define the latter as equivalence classes of strings derived by grammar and address the problem of calculating the probability of deriving a given expression with a given grammar. We show that the problem is undecidable in general. We then present specific grammars for generating linear, polynomial, and rational expressions, where algorithms for calculating the probability of a given expression exist. For those grammars, we design algorithms for calculating the exact probability and efficient approximation with arbitrary precision.