Safe and Optimal Variable Impedance Control via Certified Reinforcement Learning

TL;DR

This paper introduces Certified Gaussian Manifold Sampling, a reinforcement learning framework that guarantees stability and physical feasibility for variable impedance control in robots, enabling safe and reliable autonomous interactions.

Contribution

It presents a novel RL method that ensures Lyapunov stability and actuator feasibility by construction through sampling from a manifold of stable gain schedules.

Findings

Guarantees Lyapunov stability during policy learning

Ensures actuator feasibility without reward penalties

Demonstrates effectiveness in simulation and real robot experiments

Abstract

Reinforcement learning (RL) offers a powerful approach for robots to learn complex, collaborative skills by combining Dynamic Movement Primitives (DMPs) for motion and Variable Impedance Control (VIC) for compliant interaction. However, this model-free paradigm often risks instability and unsafe exploration due to the time-varying nature of impedance gains. This work introduces Certified Gaussian Manifold Sampling (C-GMS), a novel trajectory-centric RL framework that learns combined DMP and VIC policies while guaranteeing Lyapunov stability and actuator feasibility by construction. Our approach reframes policy exploration as sampling from a mathematically defined manifold of stable gain schedules. This ensures every policy rollout is guaranteed to be stable and physically realizable, thereby eliminating the need for reward penalties or post-hoc validation. Furthermore, we provide a theoretical guarantee that our approach ensures bounded tracking error even in the presence of bounded model errors and deployment-time uncertainties. We demonstrate the effectiveness of C-GMS in simulation and verify its efficacy on a real robot, paving the way for reliable autonomous interaction in complex environments.