Curiosity-Driven Reinforcement Learning based Low-Level Flight Control

TL;DR

This paper introduces a curiosity-driven reinforcement learning algorithm for autonomous quadcopter flight control, enabling obstacle navigation and improved exploration by leveraging prediction error as an intrinsic reward.

Contribution

It presents a novel curiosity approach based on prediction error integrated with reinforcement learning for low-level drone control.

Findings

The proposed algorithm enables the quadcopter to navigate through obstacles.

Curiosity-driven learning improves exploration and policy optimization.

The method outperforms traditional algorithms in reward maximization.

Abstract

Curiosity is one of the main motives in many of the natural creatures with measurable levels of intelligence for exploration and, as a result, more efficient learning. It makes it possible for humans and many animals to explore efficiently by searching for being in states that make them surprised with the goal of learning more about what they do not know. As a result, while being curious, they learn better. In the machine learning literature, curiosity is mostly combined with reinforcement learning-based algorithms as an intrinsic reward. This work proposes an algorithm based on the drive of curiosity for autonomous learning to control by generating proper motor speeds from odometry data. The quadcopter controlled by our proposed algorithm can pass through obstacles while controlling the Yaw direction of the quad-copter toward the desired location. To achieve that, we also propose a new curiosity approach based on prediction error. We ran tests using on-policy, off-policy, on-policy plus curiosity, and the proposed algorithm and visualized the effect of curiosity in evolving exploration patterns. Results show the capability of the proposed algorithm to learn optimal policy and maximize reward where other algorithms fail to do so.