Incorporating System-level Safety Requirements in Perception Models via Reinforcement Learning

TL;DR

This paper introduces a reinforcement learning-based training method for perception models in autonomous systems that incorporates system-level safety requirements, leading to safer system performance.

Contribution

It presents a novel approach to integrate formal safety requirements into perception model training via reinforcement learning, improving system-level safety outcomes.

Findings

Models trained with safety-aware reinforcement learning outperform baselines in safety metrics.

The approach effectively translates formal safety rules into reinforcement learning rewards.

Simulation results show enhanced safety performance in perception components.

Abstract

Perception components in autonomous systems are often developed and optimized independently of downstream decision-making and control components, relying on established performance metrics like accuracy, precision, and recall. Traditional loss functions, such as cross-entropy loss and negative log-likelihood, focus on reducing misclassification errors but fail to consider their impact on system-level safety, overlooking the varying severities of system-level failures caused by these errors. To address this limitation, we propose a novel training paradigm that augments the perception component with an understanding of system-level safety objectives. Central to our approach is the translation of system-level safety requirements, formally specified using the rulebook formalism, into safety scores. These scores are then incorporated into the reward function of a reinforcement learning framework for fine-tuning perception models with system-level safety objectives. Simulation results demonstrate that models trained with this approach outperform baseline perception models in terms of system-level safety.