Safety-Driven Deep Reinforcement Learning Framework for Cobots: A Sim2Real Approach

TL;DR

This paper introduces a safety-integrated deep reinforcement learning framework for cobots, validated through simulation and real-world testing, significantly enhancing safety and efficiency in robotic grasping tasks.

Contribution

It presents a novel Sim2Real approach embedding safety constraints directly into DRL training for cobots, improving safety compliance and operational success.

Findings

16.5% higher success rate in simulations

2.5% reduction in safety violations in real tests

Effective hazard mitigation while maintaining efficiency

Abstract

This study presents a novel methodology incorporating safety constraints into a robotic simulation during the training of deep reinforcement learning (DRL). The framework integrates specific parts of the safety requirements, such as velocity constraints, as specified by ISO 10218, directly within the DRL model that becomes a part of the robot's learning algorithm. The study then evaluated the efficiency of these safety constraints by subjecting the DRL model to various scenarios, including grasping tasks with and without obstacle avoidance. The validation process involved comprehensive simulation-based testing of the DRL model's responses to potential hazards and its compliance. Also, the performance of the system is carried out by the functional safety standards IEC 61508 to determine the safety integrity level. The study indicated a significant improvement in the safety performance of the robotic system. The proposed DRL model anticipates and mitigates hazards while maintaining operational efficiency. This study was validated in a testbed with a collaborative robotic arm with safety sensors and assessed with metrics such as the average number of safety violations, obstacle avoidance, and the number of successful grasps. The proposed approach outperforms the conventional method by a 16.5% average success rate on the tested scenarios in the simulations and 2.5% in the testbed without safety violations. The project repository is available at https://github.com/ammar-n-abbas/sim2real-ur-gym-gazebo.