Adaptive Aggregation for Safety-Critical Control

TL;DR

This paper introduces an adaptive aggregation framework that enhances safety and data efficiency in reinforcement learning for safety-critical control by transferring safety knowledge and employing safeguards.

Contribution

It proposes a novel method combining transfer learning with safety safeguards to improve safety and efficiency in reinforcement learning for real-world applications.

Findings

Fewer safety violations achieved compared to baselines.

Improved data efficiency demonstrated in experiments.

Effective transfer of safety knowledge across tasks.

Abstract

Safety has been recognized as the central obstacle to preventing the use of reinforcement learning (RL) for real-world applications. Different methods have been developed to deal with safety concerns in RL. However, learning reliable RL-based solutions usually require a large number of interactions with the environment. Likewise, how to improve the learning efficiency, specifically, how to utilize transfer learning for safe reinforcement learning, has not been well studied. In this work, we propose an adaptive aggregation framework for safety-critical control. Our method comprises two key techniques: 1) we learn to transfer the safety knowledge by aggregating the multiple source tasks and a target task through the attention network; 2) we separate the goal of improving task performance and reducing constraint violations by utilizing a safeguard. Experiment results demonstrate that our algorithm can achieve fewer safety violations while showing better data efficiency compared with several baselines.