How to ensure a safe control strategy? Towards a SRL for urban transit autonomous operation

TL;DR

This paper introduces a SSA-DRL framework combining reinforcement learning, linear temporal logic, and Monte Carlo tree search to ensure safe and optimized autonomous control in urban rail transit, addressing safety concerns during learning and execution.

Contribution

The paper presents a novel SSA-DRL framework that guarantees safety constraints and improves decision-making in urban rail transit autonomous operation.

Findings

Framework effectively meets speed and schedule constraints

Demonstrates safety and efficiency improvements over scheduled plans

Validated across sixteen different transit sections

Abstract

Deep reinforcement learning has gradually shown its latent decision-making ability in urban rail transit autonomous operation. However, since reinforcement learning can not neither guarantee safety during learning nor execution, this is still one of the major obstacles to the practical application of reinforcement learning. Given this drawback, reinforcement learning applied in the safety-critical autonomous operation domain remains challenging without generating a safe control command sequence that avoids overspeed operations. Therefore, a SSA-DRL framework is proposed in this paper for safe intelligent control of urban rail transit autonomous operation trains. The proposed framework is combined with linear temporal logic, reinforcement learning and Monte Carlo tree search and consists of four mainly module: a post-posed shielding, a searching tree module, a DRL framework and an additional actor. Furthermore, the output of the framework can meet speed constraint, schedule constraint and optimize the operation process. Finally, the proposed SSA-DRL framework for decision-making in urban rail transit autonomous operation is evaluated in sixteen different sections, and its effectiveness is demonstrated through an ablation experiment and comparison with the scheduled operation plan.