Learning from Imperfect Demonstrations via Temporal Behavior Tree-Guided Trajectory Repair

TL;DR

This paper introduces a formal framework using Temporal Behavior Trees to repair imperfect demonstrations, enabling more effective robot learning from suboptimal data.

Contribution

It presents a model-based trajectory repair method guided by TBT specifications, improving data quality for reinforcement learning without needing a kinematic model.

Findings

Repaired trajectories satisfy formal constraints and are more consistent.

The framework improves data efficiency in robot learning tasks.

Effective on both grid-world and continuous reach-avoid tasks.

Abstract

Learning robot control policies from demonstrations is a powerful paradigm, yet real-world data is often suboptimal, noisy, or otherwise imperfect, posing significant challenges for imitation and reinforcement learning. In this work, we present a formal framework that leverages Temporal Behavior Trees (TBT), an extension of Signal Temporal Logic (STL) with Behavior Tree semantics, to repair suboptimal trajectories prior to their use in downstream policy learning. Given demonstrations that violate a TBT specification, a model-based repair algorithm corrects trajectory segments to satisfy the formal constraints, yielding a dataset that is both logically consistent and interpretable. The repaired trajectories are then used to extract potential functions that shape the reward signal for reinforcement learning, guiding the agent toward task-consistent regions of the state space without requiring knowledge of the agent's kinematic model. We demonstrate the effectiveness of this framework on discrete grid-world navigation and continuous single and multi-agent reach-avoid tasks, highlighting its potential for data-efficient robot learning in settings where high-quality demonstrations cannot be assumed.