Hyperparameter Optimization for Driving Strategies Based on Reinforcement Learning

TL;DR

This paper applies Gaussian process-based Bayesian optimization to tune hyperparameters for reinforcement learning-driven autonomous driving, achieving a 4% performance improvement over manual tuning and analyzing robustness.

Contribution

It introduces an efficient hyperparameter optimization framework combining Gaussian processes, Latin hypercube sampling, and parallel evaluation for RL autonomous driving strategies.

Findings

4% performance improvement over manual tuning

Enhanced robustness and generalization of driving strategies

Effective use of Bayesian optimization in autonomous driving

Abstract

This paper focuses on hyperparameter optimization for autonomous driving strategies based on Reinforcement Learning. We provide a detailed description of training the RL agent in a simulation environment. Subsequently, we employ Efficient Global Optimization algorithm that uses Gaussian Process fitting for hyperparameter optimization in RL. Before this optimization phase, Gaussian process interpolation is applied to fit the surrogate model, for which the hyperparameter set is generated using Latin hypercube sampling. To accelerate the evaluation, parallelization techniques are employed. Following the hyperparameter optimization procedure, a set of hyperparameters is identified, resulting in a noteworthy enhancement in overall driving performance. There is a substantial increase of 4\% when compared to existing manually tuned parameters and the hyperparameters discovered during the initialization process using Latin hypercube sampling. After the optimization, we analyze the obtained results thoroughly and conduct a sensitivity analysis to assess the robustness and generalization capabilities of the learned autonomous driving strategies. The findings from this study contribute to the advancement of Gaussian process based Bayesian optimization to optimize the hyperparameters for autonomous driving in RL, providing valuable insights for the development of efficient and reliable autonomous driving systems.