Adaptive Control in Autonomous Driving via Real-Time Recurrent RL

TL;DR

This paper introduces an online reinforcement learning method using Real-Time Recurrent RL to adapt autonomous driving policies in real-time, including in real-world scenarios with event cameras.

Contribution

It extends RTRRL to support nonlinear diagonal state-space models and demonstrates online fine-tuning with event-camera data on real hardware.

Findings

LrcSSM-based policies outperform others in speed and consistency.

First demonstration of online RL fine-tuning with event cameras on standard hardware.

Effective adaptation during real-world line-following tasks.

Abstract

We study online fine-tuning of pretrained control policies for autonomous driving using Real-Time Recurrent Reinforcement Learning (RTRRL), a memory-efficient algorithm that updates policy parameters at every time step without backpropagation through time. We extend RTRRL to support LrcSSM, a recently proposed nonlinear diagonal state-space model, and combine offline behavioral cloning with online RTRRL fine-tuning to adapt policies to distribution shifts at deployment. We validate the approach in the CarRacing simulation and on a 1:10-scale RoboRacer platform equipped with an event camera, where a pretrained policy is fine-tuned online during real-world line-following. To our knowledge, this is the first demonstration of online RL fine-tuning with event-camera observations on standard (non-spiking) hardware in closed-loop control. LrcSSM-based policies improve fastest and most consistently across both settings.