Global End-Effector Pose Control of an Underactuated Aerial Manipulator via Reinforcement Learning

TL;DR

This paper presents a reinforcement learning-based control method for a lightweight, underactuated aerial manipulator to achieve precise end-effector positioning and orientation, demonstrating robustness and accuracy in real-world experiments.

Contribution

It introduces a novel reinforcement learning approach for controlling a lightweight, underactuated aerial manipulator's end-effector pose, combining simulation-trained policies with classical controllers.

Findings

Achieved centimeter-level position accuracy in flight.

Demonstrated degree-level orientation precision.

Maintained robust performance under external disturbances.

Abstract

Aerial manipulators, which combine robotic arms with multi-rotor drones, face strict constraints on arm weight and mechanical complexity. In this work, we study a lightweight 2-degree-of-freedom (DoF) arm mounted on a quadrotor via a differential mechanism, capable of full six-DoF end-effector pose control. While the minimal design enables simplicity and reduced payload, it also introduces challenges such as underactuation and sensitivity to external disturbances. To address these, we employ reinforcement learning, training a Proximal Policy Optimization (PPO) agent in simulation to generate feedforward commands for quadrotor acceleration and body rates, along with joint angle targets. These commands are tracked by an incremental nonlinear dynamic inversion (INDI) attitude controller and a PID joint controller, respectively. Flight experiments demonstrate centimeter-level position accuracy and degree-level orientation precision, with robust performance under external force disturbances, including manipulation of heavy loads and pushing tasks. The results highlight the potential of learning-based control strategies for enabling contact-rich aerial manipulation using simple, lightweight platforms. Videos of the experiment and the method are summarized in https://youtu.be/bWLTPqKcCOA.