Reinforcement Learning Enhancement Using Vector Semantic Representation and Symbolic Reasoning for Human-Centered Autonomous Emergency Braking

TL;DR

This paper introduces a neuro-symbolic feature representation and a soft logic-based reward function to enhance reinforcement learning for autonomous emergency braking, leading to safer and more context-aware decision-making in complex driving scenarios.

Contribution

It presents a novel neuro-symbolic feature extraction method combined with a Soft First-Order Logic reward function for improved reinforcement learning in autonomous driving.

Findings

Enhanced policy robustness in simulation

Improved safety metrics across traffic conditions

Better handling of occlusions and dynamic entities

Abstract

The problem with existing camera-based Deep Reinforcement Learning approaches is twofold: they rarely integrate high-level scene context into the feature representation, and they rely on rigid, fixed reward functions. To address these challenges, this paper proposes a novel pipeline that produces a neuro-symbolic feature representation that encompasses semantic, spatial, and shape information, as well as spatially boosted features of dynamic entities in the scene, with an emphasis on safety-critical road users. It also proposes a Soft First-Order Logic (SFOL) reward function that balances human values via a symbolic reasoning module. Here, semantic and spatial predicates are extracted from segmentation maps and applied to linguistic rules to obtain reward weights. Quantitative experiments in the CARLA simulation environment show that the proposed neuro-symbolic representation and SFOL reward function improved policy robustness and safety-related performance metrics compared to baseline representations and reward formulations across varying traffic densities and occlusion levels. The findings demonstrate that integrating holistic representations and soft reasoning into Reinforcement Learning can support more context-aware and value-aligned decision-making for autonomous driving.