SpecRLBench: A Benchmark for Generalization in Specification-Guided Reinforcement Learning

TL;DR

SpecRLBench is a comprehensive benchmark designed to evaluate and improve the generalization of specification-guided reinforcement learning methods across diverse tasks and environments.

Contribution

The paper introduces SpecRLBench, a new benchmark for assessing the generalization of LTL-based RL methods across multiple domains and complexities.

Findings

Existing methods struggle with unseen specifications and complex environments.

SpecRLBench reveals key challenges in generalization as task complexity increases.

The benchmark enables systematic comparison and development of more robust RL approaches.

Abstract

Specification-guided reinforcement learning (RL) provides a principled framework for encoding complex, temporally extended tasks using formal specifications such as linear temporal logic (LTL). While recent methods have shown promising results, their ability to generalize across unseen specifications and diverse environments remains insufficiently understood. In this work, we introduce SpecRLBench, a benchmark designed to evaluate the generalization capabilities of LTL-based specification-guided RL methods. The benchmark spans multiple difficulty levels across navigation and manipulation domains, incorporating both static and dynamic environments, diverse robot dynamics, and varied observation modalities. Through extensive empirical evaluation, we characterize the strengths and limitations of existing approaches and reveal the challenges that emerge as specification and environment complexity increase. SpecRLBench provides a structured platform for systematic comparison and supports the development of more generalizable specification-guided RL methods. Code is available at https://github.com/BU-DEPEND-Lab/SpecRLBench.