Live LTL Progress Tracking: Towards Task-Based Exploration
Noel Brindise, Cedric Langbort, Melkior Ornik

TL;DR
This paper introduces a framework for tracking the progress of reinforcement learning agents on complex tasks specified by linear temporal logic, enabling better exploration and reward shaping.
Contribution
The paper presents a novel Live LTL Progress Tracking framework that encodes task progress in RL using a vector updated at each step, facilitating new metrics and exploration strategies.
Findings
Framework formalized and demonstrated with a simple example
Tracking vector encodes detailed task execution information
Potential for integration into RL for improved exploration and reward shaping
Abstract
Motivated by the challenge presented by non-Markovian objectives in reinforcement learning (RL), we present a novel framework to track and represent the progress of autonomous agents through complex, multi-stage tasks. Given a specification in finite linear temporal logic (LTL), the framework establishes a 'tracking vector' which updates at each time step in a trajectory rollout. The values of the vector represent the status of the specification as the trajectory develops, assigning true, false, or 'open' labels (where 'open' is used for indeterminate cases). Applied to an LTL formula tree, the tracking vector can be used to encode detailed information about how a task is executed over a trajectory, providing a potential tool for new performance metrics, diverse exploration, and reward shaping. In this paper, we formally present the framework and algorithm, collectively named Live LTL…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
