BF++: a language for general-purpose program synthesis

TL;DR

BF++ is a new programming language designed for automatic agent programming in POMDPs, enabling neural program synthesis to improve decision-making in reinforcement learning tasks, especially where expert knowledge integration is crucial.

Contribution

The paper introduces BF++, a novel language tailored for program synthesis in decision systems, bridging the gap between neural models and symbolic, expert-validated programs.

Findings

Successfully applied neural program synthesis to OpenAI Gym benchmarks.

Demonstrated improved incorporation of expert knowledge into decision models.

Showed potential for more transparent and reviewable decision mechanisms.

Abstract

Most state of the art decision systems based on Reinforcement Learning (RL) are data-driven black-box neural models, where it is often difficult to incorporate expert knowledge into the models or let experts review and validate the learned decision mechanisms. Knowledge-insertion and model review are important requirements in many applications involving human health and safety. One way to bridge the gap between data and knowledge driven systems is program synthesis: replacing a neural network that outputs decisions with a symbolic program generated by a neural network or by means of genetic programming. We propose a new programming language, BF++, designed specifically for automatic programming of agents in a Partially Observable Markov Decision Process (POMDP) setting and apply neural program synthesis to solve standard OpenAI Gym benchmarks.