Learning Extreme Hummingbird Maneuvers on Flapping Wing Robots

TL;DR

This paper presents a hybrid control strategy combining nonlinear control and reinforcement learning to enable a small robotic hummingbird to perform rapid, extreme maneuvers similar to those observed in real hummingbirds, with successful real-world implementation.

Contribution

The authors develop and experimentally validate a hybrid control policy that achieves hummingbird-like evasive maneuvers on a small robotic platform, bridging model-based and model-free control methods.

Findings

Achieved near-real hummingbird evasive maneuvers in simulation and on a physical robot.

Demonstrated successful transfer of control policy from simulation to real robot.

Enabled rapid maneuvering within 0.2 seconds, mimicking biological performance.

Abstract

Biological studies show that hummingbirds can perform extreme aerobatic maneuvers during fast escape. Given a sudden looming visual stimulus at hover, a hummingbird initiates a fast backward translation coupled with a 180-degree yaw turn, which is followed by instant posture stabilization in just under 10 wingbeats. Consider the wingbeat frequency of 40Hz, this aggressive maneuver is carried out in just 0.2 seconds. Inspired by the hummingbirds' near-maximal performance during such extreme maneuvers, we developed a flight control strategy and experimentally demonstrated that such maneuverability can be achieved by an at-scale 12-gram hummingbird robot equipped with just two actuators. The proposed hybrid control policy combines model-based nonlinear control with model-free reinforcement learning. We use model-based nonlinear control for nominal flight control, as the dynamic model is relatively accurate for these conditions. However, during extreme maneuver, the modeling error becomes unmanageable. A model-free reinforcement learning policy trained in simulation was optimized to 'destabilize' the system and maximize the performance during maneuvering. The hybrid policy manifests a maneuver that is close to that observed in hummingbirds. Direct simulation-to-real transfer is achieved, demonstrating the hummingbird-like fast evasive maneuvers on the at-scale hummingbird robot.