Numeric Reward Machines
Kristina Levina, Nikolaos Pappas, Athanasios Karapantelakis, Aneta, Vulgarakis Feljan, Jendrik Seipp

TL;DR
This paper extends reward machines in reinforcement learning to handle numeric features like distances, enabling more effective guidance in inherently numeric tasks, and demonstrates significant performance improvements in the Craft domain.
Contribution
The paper introduces two types of reward machines that incorporate numeric features, expanding their applicability to numeric tasks in reinforcement learning.
Findings
Numeric reward machines outperform baseline in Craft domain.
Numeric features improve learning efficiency.
Approaches generalize to various numeric features.
Abstract
Reward machines inform reinforcement learning agents about the reward structure of the environment and often drastically speed up the learning process. However, reward machines only accept Boolean features such as robot-reached-gold. Consequently, many inherently numeric tasks cannot profit from the guidance offered by reward machines. To address this gap, we aim to extend reward machines with numeric features such as distance-to-gold. For this, we present two types of reward machines: numeric-Boolean and numeric. In a numeric-Boolean reward machine, distance-to-gold is emulated by two Boolean features distance-to-gold-decreased and robot-reached-gold. In a numeric reward machine, distance-to-gold is used directly alongside the Boolean feature robot-reached-gold. We compare our new approaches to a baseline reward machine in the Craft domain, where the numeric feature is the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsReceptor Mechanisms and Signaling
MethodsSPEED: Separable Pyramidal Pooling EncodEr-Decoder for Real-Time Monocular Depth Estimation on Low-Resource Settings · Q-Learning
