Learning to Race through Coordinate Descent Bayesian Optimisation

TL;DR

This paper introduces a coordinate descent Bayesian optimization method to efficiently learn control policies for autonomous racing, minimizing lap time with minimal real-world interactions.

Contribution

It proposes a novel BO approach that optimizes high-dimensional control policies sequentially, improving efficiency in complex, real-time robotic tasks.

Findings

The method outperforms existing optimization techniques in simulated racing scenarios.

It effectively reduces the number of evaluations needed to find optimal policies.

The approach demonstrates robustness despite limited prior information.

Abstract

In the automation of many kinds of processes, the observable outcome can often be described as the combined effect of an entire sequence of actions, or controls, applied throughout its execution. In these cases, strategies to optimise control policies for individual stages of the process might not be applicable, and instead the whole policy might have to be optimised at once. On the other hand, the cost to evaluate the policy's performance might also be high, being desirable that a solution can be found with as few interactions as possible with the real system. We consider the problem of optimising control policies to allow a robot to complete a given race track within a minimum amount of time. We assume that the robot has no prior information about the track or its own dynamical model, just an initial valid driving example. Localisation is only applied to monitor the robot and to provide an indication of its position along the track's centre axis. We propose a method for finding a policy that minimises the time per lap while keeping the vehicle on the track using a Bayesian optimisation (BO) approach over a reproducing kernel Hilbert space. We apply an algorithm to search more efficiently over high-dimensional policy-parameter spaces with BO, by iterating over each dimension individually, in a sequential coordinate descent-like scheme. Experiments demonstrate the performance of the algorithm against other methods in a simulated car racing environment.