Autonomous overtaking trajectory optimization using reinforcement learning and opponent pose estimation

TL;DR

This paper introduces a reinforcement learning-based system utilizing LiDAR and depth camera data for autonomous overtaking, achieving safe and efficient maneuvers in racing scenarios.

Contribution

It develops a multi-agent reinforcement learning approach with sensor fusion for optimal overtaking trajectory planning in autonomous vehicles.

Findings

Successful overtaking in simulation and real-world tests

Pose estimation RMSE of 0.0816 m in x and 0.0531 m in y

Effective sensor fusion using UKF improves trajectory accuracy

Abstract

Vehicle overtaking is one of the most complex driving maneuvers for autonomous vehicles. To achieve optimal autonomous overtaking, driving systems rely on multiple sensors that enable safe trajectory optimization and overtaking efficiency. This paper presents a reinforcement learning mechanism for multi-agent autonomous racing environments, enabling overtaking trajectory optimization, based on LiDAR and depth image data. The developed reinforcement learning agent uses pre-generated raceline data and sensor inputs to compute the steering angle and linear velocity for optimal overtaking. The system uses LiDAR with a 2D detection algorithm and a depth camera with YOLO-based object detection to identify the vehicle to be overtaken and its pose. The LiDAR and the depth camera detection data are fused using a UKF for improved opponent pose estimation and trajectory optimization for overtaking in racing scenarios. The results show that the proposed algorithm successfully performs overtaking maneuvers in both simulation and real-world experiments, with pose estimation RMSE of (0.0816, 0.0531) m in (x, y).