On the Verge of Solving Rocket League using Deep Reinforcement Learning and Sim-to-sim Transfer

TL;DR

This paper demonstrates that deep reinforcement learning agents trained in a simulation can be effectively transferred to the actual Rocket League game, achieving high success rates in goalkeeping and striking tasks.

Contribution

It introduces a sim-to-sim transfer approach for training Rocket League agents, showing successful transfer of learned behaviors from simulation to the real game.

Findings

Goalkeeper agent saves nearly 100% of shots after transfer.

Striker agent scores in about 75% of cases after transfer.

Training in simulation is effective despite some inaccuracies.

Abstract

Autonomously trained agents that are supposed to play video games reasonably well rely either on fast simulation speeds or heavy parallelization across thousands of machines running concurrently. This work explores a third way that is established in robotics, namely sim-to-real transfer, or if the game is considered a simulation itself, sim-to-sim transfer. In the case of Rocket League, we demonstrate that single behaviors of goalies and strikers can be successfully learned using Deep Reinforcement Learning in the simulation environment and transferred back to the original game. Although the implemented training simulation is to some extent inaccurate, the goalkeeping agent saves nearly 100% of its faced shots once transferred, while the striking agent scores in about 75% of cases. Therefore, the trained agent is robust enough and able to generalize to the target domain of Rocket League.