Transfer and Online Reinforcement Learning in STT-MRAM Based Embedded Systems for Autonomous Drones

TL;DR

This paper proposes a hardware-software co-design for autonomous drones that enables real-time reinforcement learning using a hybrid memory system, significantly reducing energy consumption while maintaining performance.

Contribution

It introduces a transfer learning and reinforcement learning framework tailored for NVM-based embedded systems in drones, optimizing memory usage and energy efficiency.

Findings

Achieved 83.4% lower energy per image frame.

Enabled real-time RL updates on SRAM for improved responsiveness.

Maintained comparable performance to end-to-end RL methods.

Abstract

In this paper we present an algorithm-hardware codesign for camera-based autonomous flight in small drones. We show that the large write-latency and write-energy for nonvolatile memory (NVM) based embedded systems makes them unsuitable for real-time reinforcement learning (RL). We address this by performing transfer learning (TL) on metaenvironments and RL on the last few layers of a deep convolutional network. While the NVM stores the meta-model from TL, an on-die SRAM stores the weights of the last few layers. Thus all the real-time updates via RL are carried out on the SRAM arrays. This provides us with a practical platform with comparable performance as end-to-end RL and 83.4% lower energy per image frame