Risk-Sensitive Inhibitory Control for Safe Reinforcement Learning

TL;DR

This paper introduces a risk-sensitive inhibitory control method for reinforcement learning that ensures safety by incorporating risk attitudes into value functions, demonstrated through simulation results.

Contribution

It proposes a novel risk-aware safety condition for reinforcement learning, integrating human-like risk attitudes into inhibitory control for improved safety guarantees.

Findings

Successfully guarantees state constraints in simulations

Effectively incorporates risk attitudes into safety control

Provides a learning framework for risk-sensitive value functions

Abstract

Humans have the ability to deviate from their natural behavior when necessary, which is a cognitive process called response inhibition. Similar approaches have independently received increasing attention in recent years for ensuring the safety of control. Realized using control barrier functions or predictive safety filters, these approaches can effectively ensure the satisfaction of state constraints through an online adaptation of nominal control laws, e.g., obtained through reinforcement learning. While the focus of these realizations of inhibitory control has been on risk-neutral formulations, human studies have shown a tight link between response inhibition and risk attitude. Inspired by this insight, we propose a flexible, risk-sensitive method for inhibitory control. Our method is based on a risk-aware condition for value functions, which guarantees the satisfaction of state constraints. We propose a method for learning these value functions using common techniques from reinforcement learning and derive sufficient conditions for its success. By enforcing the derived safety conditions online using the learned value function, risk-sensitive inhibitory control is effectively achieved. The effectiveness of the developed control scheme is demonstrated in simulations.