Generalization of Compositional Tasks with Logical Specification via Implicit Planning
Duo Xu, Faramarz Fekri

TL;DR
This paper proposes a hierarchical reinforcement learning framework with an implicit planner that improves the generalization, efficiency, and optimality of policies for compositional tasks defined by logical specifications, addressing challenges in long-horizon, sub-task dependent scenarios.
Contribution
It introduces a novel hierarchical RL approach with an implicit planner using GNNs for planning in latent space, enhancing generalization and performance in compositional tasks.
Findings
Outperforms previous methods in efficiency
Achieves better task generalization
Demonstrates improved policy optimality
Abstract
In this study, we address the challenge of learning generalizable policies for compositional tasks defined by logical specifications. These tasks consist of multiple temporally extended sub-tasks. Due to the sub-task inter-dependencies and sparse reward issue in long-horizon tasks, existing reinforcement learning (RL) approaches, such as task-conditioned and goal-conditioned policies, continue to struggle with slow convergence and sub-optimal performance in generalizing to compositional tasks. To overcome these limitations, we introduce a new hierarchical RL framework that enhances the efficiency and optimality of task generalization. At the high level, we present an implicit planner specifically designed for generalizing compositional tasks. This planner selects the next sub-task and estimates the multi-step return for completing the remaining task to complete from the current state.…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAI-based Problem Solving and Planning · Logic, programming, and type systems · Logic, Reasoning, and Knowledge
MethodsGraph Neural Network
