Autonomous Reinforcement Learning Robot Control with Intel's Loihi 2 Neuromorphic Hardware

TL;DR

This paper demonstrates converting reinforcement learning policies into spiking neural networks for deployment on Intel's Loihi 2 neuromorphic hardware, enabling low-latency, energy-efficient robotic control in simulation.

Contribution

It introduces an end-to-end pipeline for transforming RL-trained ANNs into SDNNs compatible with Loihi 2, facilitating neuromorphic deployment for robotics.

Findings

SDNNs can be effectively deployed on Loihi 2 hardware.

Neuromorphic implementation achieves comparable control performance to GPU-based systems.

Energy efficiency and low latency are demonstrated in robotic control tasks.

Abstract

We present an end-to-end pipeline for deploying reinforcement learning (RL) trained Artificial Neural Networks (ANNs) on neuromorphic hardware by converting them into spiking Sigma-Delta Neural Networks (SDNNs). We demonstrate that an ANN policy trained entirely in simulation can be transformed into an SDNN compatible with Intel's Loihi 2 architecture, enabling low-latency and energy-efficient inference. As a test case, we use an RL policy for controlling the Astrobee free-flying robot, similar to a previously hardware in space-validated controller. The policy, trained with Rectified Linear Units (ReLUs), is converted to an SDNN and deployed on Intel's Loihi 2, then evaluated in NVIDIA's Omniverse Isaac Lab simulation environment for closed-loop control of Astrobee's motion. We compare execution performance between GPU and Loihi 2. The results highlight the feasibility of using neuromorphic platforms for robotic control and establish a pathway toward energy-efficient, real-time neuromorphic computation in future space and terrestrial robotics applications.