Reliability Quantification of Deep Reinforcement Learning-based Control

TL;DR

This paper introduces a novel neural network-based method for quantifying the reliability of deep reinforcement learning control systems, demonstrated on DQN, to enhance safety and performance in critical applications.

Contribution

A new reliability quantification approach using reference and evaluator networks, addressing issues in existing methods and improving DRL control safety.

Findings

Effective reliability evaluation on DQN control.

Improved control performance by model switching based on reliability.

Validated method's effectiveness in safety-critical scenarios.

Abstract

Reliability quantification of deep reinforcement learning (DRL)-based control is a significant challenge for the practical application of artificial intelligence (AI) in safety-critical systems. This study proposes a method for quantifying the reliability of DRL-based control. First, an existing method, random noise distillation, was applied to the reliability evaluation to clarify the issues to be solved. Second, a novel method for reliability quantification was proposed to solve these issues. The reliability is quantified using two neural networks: reference and evaluator. They have the same structure with the same initial parameters. The outputs of the two networks were the same before training. During training, the evaluator network parameters were updated to maximize the difference between the reference and evaluator networks for trained data. Thus, the reliability of the DRL-based control for a state can be evaluated based on the difference in output between the two networks. The proposed method was applied to DQN-based control as an example of a simple task, and its effectiveness was demonstrated. Finally, the proposed method was applied to the problem of switching trained models depending on the state. Con-sequently, the performance of the DRL-based control was improved by switching the trained models according to their reliability.