RLang: A Declarative Language for Describing Partial World Knowledge to Reinforcement Learning Agents

TL;DR

RLang is a new domain-specific language that allows detailed specification of partial world knowledge for reinforcement learning agents, enabling improved integration of domain insights across various RL methods.

Contribution

It introduces RLang, a DSL with precise syntax and semantics for expressing comprehensive partial world knowledge to RL agents, unlike existing languages limited to single elements.

Findings

RLang can specify knowledge about all elements of an MDP.

RLang programs can be exploited by diverse RL algorithms.

Demonstrated effectiveness across multiple RL methods.

Abstract

We introduce RLang, a domain-specific language (DSL) for communicating domain knowledge to an RL agent. Unlike existing RL DSLs that ground to \textit{single} elements of a decision-making formalism (e.g., the reward function or policy), RLang can specify information about every element of a Markov decision process. We define precise syntax and grounding semantics for RLang, and provide a parser that grounds RLang programs to an algorithm-agnostic \textit{partial} world model and policy that can be exploited by an RL agent. We provide a series of example RLang programs demonstrating how different RL methods can exploit the resulting knowledge, encompassing model-free and model-based tabular algorithms, policy gradient and value-based methods, hierarchical approaches, and deep methods.