Challenging On Car Racing Problem from OpenAI gym

TL;DR

This paper compares evolutionary algorithms and deep Q-learning for solving the challenging OpenAI gym car racing task, highlighting their convergence speed and efficiency under limited hardware resources.

Contribution

It introduces and evaluates two approaches—genetic multi-layer perceptron and double deep Q-learning—for pixel-based continuous control in car racing.

Findings

Genetic multi-layer perceptron converges faster.

Double deep Q-learning achieves higher scores with more episodes.

Genetic methods can be more hardware-efficient.

Abstract

This project challenges the car racing problem from OpenAI gym environment. The problem is very challenging since it requires computer to finish the continuous control task by learning from pixels. To tackle this challenging problem, we explored two approaches including evolutionary algorithm based genetic multi-layer perceptron and double deep Q-learning network. The result shows that the genetic multi-layer perceptron can converge fast but when training many episodes, double deep Q-learning can get better score. We analyze the result and draw a conclusion that for limited hardware resources, using genetic multi-layer perceptron sometimes can be more efficient.