Learning-based vs Model-free Adaptive Control of a MAV under Wind Gust

TL;DR

This paper compares learning-based and model-free adaptive control methods for a micro aerial vehicle under wind gusts, demonstrating the potential of learning-based approaches through realistic simulations.

Contribution

It introduces a simple learning-based adaptive control method using deep reinforcement learning and compares it with a model-free approach in MAV wind disturbance scenarios.

Findings

Learning-based control outperforms model-free in simulation.

Deep reinforcement learning effectively tunes the controller.

Learning-based methods show promise for real-world uncertainties.

Abstract

Navigation problems under unknown varying conditions are among the most important and well-studied problems in the control field. Classic model-based adaptive control methods can be applied only when a convenient model of the plant or environment is provided. Recent model-free adaptive control methods aim at removing this dependency by learning the physical characteristics of the plant and/or process directly from sensor feedback. Although there have been prior attempts at improving these techniques, it remains an open question as to whether it is possible to cope with real-world uncertainties in a control system that is fully based on either paradigm. We propose a conceptually simple learning-based approach composed of a full state feedback controller, tuned robustly by a deep reinforcement learning framework based on the Soft Actor-Critic algorithm. We compare it, in realistic simulations, to a model-free controller that uses the same deep reinforcement learning framework for the control of a micro aerial vehicle under wind gust. The results indicate the great potential of learning-based adaptive control methods in modern dynamical systems.