QuadSim: A Quadcopter Rotational Dynamics Simulation Framework For Reinforcement Learning Algorithms

TL;DR

QuadSim is a versatile, mathematically based quadcopter simulation framework designed for testing reinforcement learning algorithms with flexible configurations, noise models, and compatibility with OpenAI Gym, supporting parallel processing.

Contribution

The paper introduces QuadSim, a novel simulation framework that supports both linear and nonlinear quadcopter models, noise addition, and integration with OpenAI Gym for RL research.

Findings

Deep RL algorithms successfully trained in QuadSim

Framework supports deterministic and stochastic simulations

Parallel processing enhances training efficiency

Abstract

This study focuses on designing and developing a mathematically based quadcopter rotational dynamics simulation framework for testing reinforcement learning (RL) algorithms in many flexible configurations. The design of the simulation framework aims to simulate both linear and nonlinear representations of a quadcopter by solving initial value problems for ordinary differential equation (ODE) systems. In addition, the simulation environment is capable of making the simulation deterministic/stochastic by adding random Gaussian noise in the forms of process and measurement noises. In order to ensure that the scope of this simulation environment is not limited only with our own RL algorithms, the simulation environment has been expanded to be compatible with the OpenAI Gym toolkit. The framework also supports multiprocessing capabilities to run simulation environments simultaneously in parallel. To test these capabilities, many state-of-the-art deep RL algorithms were trained in this simulation framework and the results were compared in detail.