A Neural Network Mode for PX4 on Embedded Flight Controllers

TL;DR

This paper presents an open-source neural network controller framework integrated into the PX4 autopilot, enabling end-to-end position tracking on embedded flight controllers for drones, with demonstrated simulation and real-world performance.

Contribution

It introduces a novel, open-source pipeline for deploying neural network control policies directly on PX4 flight controllers, combining simulation-trained policies with embedded inference.

Findings

Neural network policies achieve similar tracking performance in simulation and real-world tests.

The framework simplifies deploying learning-based control on embedded hardware.

Open-source tools facilitate further research and development in neural flight control.

Abstract

This paper contributes an open-sourced implementation of a neural-network based controller framework within the PX4 stack. We develop a custom module for inference on the microcontroller while retaining all of the functionality of the PX4 autopilot. Policies trained in the Aerial Gym Simulator are converted to the TensorFlow Lite format and then built together with PX4 and flashed to the flight controller. The policies substitute the control-cascade within PX4 to offer an end-to-end position-setpoint tracking controller directly providing normalized motor RPM setpoints. Experiments conducted in simulation and the real-world show similar tracking performance. We thus provide a flight-ready pipeline for testing neural control policies in the real world. The pipeline simplifies the deployment of neural networks on embedded flight controller hardware thereby accelerating research on learning-based control. Both the Aerial Gym Simulator and the PX4 module are open-sourced at https://github.com/ntnu-arl/aerial_gym_simulator and https://github.com/SindreMHegre/PX4-Autopilot-public/tree/for_paper. Video: https://youtu.be/lY1OKz_UOqM?si=VtzL243BAY3lblTJ.