SafeGen-LLM: Enhancing Safety Generalization in Task Planning for Robotic Systems

TL;DR

SafeGen-LLM is a novel large language model designed to improve safety adherence and generalization in robotic task planning across multiple domains and input formats, addressing limitations of classical and RL-based methods.

Contribution

The paper introduces SafeGen-LLM, a two-stage training framework that enhances safety compliance and generalization in LLM-based robotic planning, using a new multi-domain safety benchmark and formal verification-guided training.

Findings

SafeGen-LLM outperforms existing methods in safety adherence across multiple domains.

The two-stage training improves safety generalization and planning accuracy.

SafeGen-LLM handles various input formats effectively.

Abstract

Safety-critical task planning in robotic systems remains challenging: classical planners suffer from poor scalability, Reinforcement Learning (RL)-based methods generalize poorly, and base Large Language Models (LLMs) cannot guarantee safety. To address this gap, we propose safety-generalizable large language models, named SafeGen-LLM. SafeGen-LLM can not only enhance the safety satisfaction of task plans but also generalize well to novel safety properties in various domains. We first construct a multi-domain Planning Domain Definition Language 3 (PDDL3) benchmark with explicit safety constraints. Then, we introduce a two-stage post-training framework: Supervised Fine-Tuning (SFT) on a constraint-compliant planning dataset to learn planning syntax and semantics, and Group Relative Policy Optimization (GRPO) guided by fine-grained reward machines derived from formal verification to enforce safety alignment and by curriculum learning to better handle complex tasks. Extensive experiments show that SafeGen-LLM achieves strong safety generalization and outperforms frontier proprietary baselines across multi-domain planning tasks and multiple input formats (e.g., PDDLs and natural language).