Expert Human-Level Driving in Gran Turismo Sport Using Deep Reinforcement Learning with Image-based Representation

TL;DR

This paper presents a vision-based deep reinforcement learning approach for realistic racing in Gran Turismo Sport, achieving human-level performance using only game screen images and surpassing built-in AI in time trials.

Contribution

The study introduces a novel image-based control algorithm that replaces traditional environment measurements, enabling expert-level driving in a high-fidelity racing simulator.

Findings

Achieves human-level driving performance using only visual inputs.

Outperforms built-in AI in time trial tasks.

Scores among the top 10% of approximately 28,000 human players.

Abstract

When humans play virtual racing games, they use visual environmental information on the game screen to understand the rules within the environments. In contrast, a state-of-the-art realistic racing game AI agent that outperforms human players does not use image-based environmental information but the compact and precise measurements provided by the environment. In this paper, a vision-based control algorithm is proposed and compared with human player performances under the same conditions in realistic racing scenarios using Gran Turismo Sport (GTS), which is known as a high-fidelity realistic racing simulator. In the proposed method, the environmental information that constitutes part of the observations in conventional state-of-the-art methods is replaced with feature representations extracted from game screen images. We demonstrate that the proposed method performs expert human-level vehicle control under high-speed driving scenarios even with game screen images as high-dimensional inputs. Additionally, it outperforms the built-in AI in GTS in a time trial task, and its score places it among the top 10% approximately 28,000 human players.