Omega-Regular Reward Machines
Ernst Moritz Hahn, Mateo Perez, Sven Schewe, Fabio Somenzi, Ashutosh, Trivedi, Dominik Wojtczak

TL;DR
This paper introduces omega-regular reward machines, combining reward machines with omega-regular languages to create expressive reward mechanisms for reinforcement learning, supported by a new algorithm and experimental validation.
Contribution
It proposes omega-regular reward machines and a model-free RL algorithm to handle complex non-Markovian rewards in reinforcement learning.
Findings
The algorithm computes epsilon-optimal strategies effectively.
Experimental results demonstrate the approach's practicality.
Omega-regular reward machines enhance reward expressiveness.
Abstract
Reinforcement learning (RL) is a powerful approach for training agents to perform tasks, but designing an appropriate reward mechanism is critical to its success. However, in many cases, the complexity of the learning objectives goes beyond the capabilities of the Markovian assumption, necessitating a more sophisticated reward mechanism. Reward machines and omega-regular languages are two formalisms used to express non-Markovian rewards for quantitative and qualitative objectives, respectively. This paper introduces omega-regular reward machines, which integrate reward machines with omega-regular languages to enable an expressive and effective reward mechanism for RL. We present a model-free RL algorithm to compute epsilon-optimal strategies against omega-egular reward machines and evaluate the effectiveness of the proposed algorithm through experiments.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsReceptor Mechanisms and Signaling · Reinforcement Learning in Robotics · Topic Modeling
