RAMario: Experimental Approach to Reptile Algorithm -- Reinforcement Learning for Mario

TL;DR

This paper explores using the Reptile meta-learning algorithm to train a neural network for playing Super Mario Bros, comparing its performance with PPO and DQN, and demonstrating promising results in few-shot learning scenarios.

Contribution

It introduces an experimental application of Reptile to reinforcement learning in video games, showing competitive performance against established algorithms.

Findings

Reptile achieved a total distance of ~2300 in the game environment.

Reptile outperformed PPO and DQN in certain metrics.

The approach demonstrates potential for few-shot learning in game AI.

Abstract

This research paper presents an experimental approach to using the Reptile algorithm for reinforcement learning to train a neural network to play Super Mario Bros. We implement the Reptile algorithm using the Super Mario Bros Gym library and TensorFlow in Python, creating a neural network model with a single convolutional layer, a flatten layer, and a dense layer. We define the optimizer and use the Reptile class to create an instance of the Reptile meta-learning algorithm. We train the model using multiple tasks and episodes, choosing actions using the current weights of the neural network model, taking those actions in the environment, and updating the model weights using the Reptile algorithm. We evaluate the performance of the algorithm by printing the total reward for each episode. In addition, we compare the performance of the Reptile algorithm approach to two other popular reinforcement learning algorithms, Proximal Policy Optimization (PPO) and Deep Q-Network (DQN), applied to the same Super Mario Bros task. Our results demonstrate that the Reptile algorithm provides a promising approach to few-shot learning in video game AI, with comparable or even better performance than the other two algorithms, particularly in terms of moves vs distance that agent performs for 1M episodes of training. The results shows that best total distance for world 1-2 in the game environment were ~1732 (PPO), ~1840 (DQN) and ~2300 (RAMario). Full code is available at https://github.com/s4nyam/RAMario.