A Value Function Space Approach for Hierarchical Planning with Signal Temporal Logic Tasks

TL;DR

This paper introduces a hierarchical planning framework using Value Function Space to efficiently synthesize skill sequences for Signal Temporal Logic tasks, reducing training needs in complex environments.

Contribution

It proposes a novel hierarchical approach that constructs a Value Function Space for abstraction and employs neural networks and sampling to plan STL tasks without additional low-level training.

Findings

Successfully accomplishes STL tasks in Safety Gym and ManiSkill environments.

Reduces training burdens by avoiding further low-level environment training.

Demonstrates effectiveness of the VFS approach in complex planning scenarios.

Abstract

Signal Temporal Logic (STL) has emerged as an expressive language for reasoning intricate planning objectives. However, existing STL-based methods often assume full observation and known dynamics, which imposes constraints on real-world applications. To address this challenge, we propose a hierarchical planning framework that starts by constructing the Value Function Space (VFS) for state and action abstraction, which embeds functional information about affordances of the low-level skills. Subsequently, we utilize a neural network to approximate the dynamics in the VFS and employ sampling based optimization to synthesize high-level skill sequences that maximize the robustness measure of the given STL tasks in the VFS. Then those skills are executed in the low-level environment. Empirical evaluations in the Safety Gym and ManiSkill environments demonstrate that our method accomplish the STL tasks without further training in the low-level environments, substantially reducing the training burdens.