Neurosymbolic Meta-Reinforcement Lookahead Learning Achieves Safe Self-Driving in Non-Stationary Environments

TL;DR

This paper introduces NUMERLA, a neurosymbolic meta-reinforcement learning algorithm that enables self-driving cars to adapt in real-time while maintaining safety in unpredictable urban environments.

Contribution

The paper presents a novel neurosymbolic meta-reinforcement learning approach with lookahead constraints for safe, adaptive self-driving in non-stationary settings.

Findings

NUMERLA achieves real-time safety and adaptability in urban driving scenarios.

The method outperforms baseline algorithms in safety metrics.

NUMERLA maintains safety during rapid environmental changes.

Abstract

In the area of learning-driven artificial intelligence advancement, the integration of machine learning (ML) into self-driving (SD) technology stands as an impressive engineering feat. Yet, in real-world applications outside the confines of controlled laboratory scenarios, the deployment of self-driving technology assumes a life-critical role, necessitating heightened attention from researchers towards both safety and efficiency. To illustrate, when a self-driving model encounters an unfamiliar environment in real-time execution, the focus must not solely revolve around enhancing its anticipated performance; equal consideration must be given to ensuring its execution or real-time adaptation maintains a requisite level of safety. This study introduces an algorithm for online meta-reinforcement learning, employing lookahead symbolic constraints based on \emph{Neurosymbolic Meta-Reinforcement Lookahead Learning} (NUMERLA). NUMERLA proposes a lookahead updating mechanism that harmonizes the efficiency of online adaptations with the overarching goal of ensuring long-term safety. Experimental results demonstrate NUMERLA confers the self-driving agent with the capacity for real-time adaptability, leading to safe and self-adaptive driving under non-stationary urban human-vehicle interaction scenarios.