PyFlyt -- UAV Simulation Environments for Reinforcement Learning Research

TL;DR

PyFlyt is a flexible, standardized simulation platform built on Bullet physics for testing reinforcement learning algorithms on UAVs, supporting various configurations, tasks, and reward settings to advance UAV control research.

Contribution

Introduces PyFlyt, a modular, open-source UAV simulation environment with native Gymnasium API support, enabling standardized testing of RL algorithms on diverse UAV models.

Findings

RL agents successfully trained on quadrotor and fixed-wing UAVs

Agents can learn in sparse reward environments

PyFlyt demonstrates the effectiveness of RL in UAV control

Abstract

Unmanned aerial vehicles (UAVs) have numerous applications, but their efficient and optimal flight can be a challenge. Reinforcement Learning (RL) has emerged as a promising approach to address this challenge, yet there is no standardized library for testing and benchmarking RL algorithms on UAVs. In this paper, we introduce PyFlyt, a platform built on the Bullet physics engine with native Gymnasium API support. PyFlyt provides modular implementations of simple components, such as motors and lifting surfaces, allowing for the implementation of UAVs of arbitrary configurations. Additionally, PyFlyt includes various task definitions and multiple reward function settings for each vehicle type. We demonstrate the effectiveness of PyFlyt by training various RL agents for two UAV models: quadrotor and fixed-wing. Our findings highlight the effectiveness of RL in UAV control and planning, and further show that it is possible to train agents in sparse reward settings for UAVs. PyFlyt fills a gap in existing literature by providing a flexible and standardised platform for testing RL algorithms on UAVs. We believe that this will inspire more standardised research in this direction.