Self-Adaptive Driving in Nonstationary Environments through Conjectural Online Lookahead Adaptation

TL;DR

This paper introduces COLA, an online meta reinforcement learning algorithm that enables self-driving cars to adapt in real-time to changing weather and lighting conditions, improving robustness over traditional policies.

Contribution

The work presents a novel online meta RL method called COLA that maximizes future performance conjectures for real-time adaptation in nonstationary environments.

Findings

COLA outperforms baseline policies in dynamic weather and lighting conditions.

The approach demonstrates effective online adaptability in self-driving scenarios.

Experimental results validate the superiority of COLA in nonstationary environments.

Abstract

Powered by deep representation learning, reinforcement learning (RL) provides an end-to-end learning framework capable of solving self-driving (SD) tasks without manual designs. However, time-varying nonstationary environments cause proficient but specialized RL policies to fail at execution time. For example, an RL-based SD policy trained under sunny days does not generalize well to rainy weather. Even though meta learning enables the RL agent to adapt to new tasks/environments, its offline operation fails to equip the agent with online adaptation ability when facing nonstationary environments. This work proposes an online meta reinforcement learning algorithm based on the \emph{conjectural online lookahead adaptation} (COLA). COLA determines the online adaptation at every step by maximizing the agent's conjecture of the future performance in a lookahead horizon. Experimental results demonstrate that under dynamically changing weather and lighting conditions, the COLA-based self-adaptive driving outperforms the baseline policies in terms of online adaptability. A demo video, source code, and appendixes are available at {\tt https://github.com/Panshark/COLA}