Fully Spiking Actor-Critic Neural Network for Robotic Manipulation

TL;DR

This paper introduces a simplified fully spiking neural network-based reinforcement learning framework for robotic manipulation, emphasizing energy efficiency and fast inference, validated through simulation experiments.

Contribution

It presents a novel hybrid curriculum RL framework using a minimalistic SNN architecture combined with PPO, enhancing efficiency and resource suitability for robotic tasks.

Findings

Achieved superior task performance in simulation.

Demonstrated reduced energy consumption compared to ANNs.

Validated scalability and efficiency in dynamic environments.

Abstract

This study proposes a hybrid curriculum reinforcement learning (CRL) framework based on a fully spiking neural network (SNN) for 9-degree-of-freedom robotic arms performing target reaching and grasping tasks. To reduce network complexity and inference latency, the SNN architecture is simplified to include only an input and an output layer, which shows strong potential for resource-constrained environments. Building on the advantages of SNNs-high inference speed, low energy consumption, and spike-based biological plausibility, a temporal progress-partitioned curriculum strategy is integrated with the Proximal Policy Optimization (PPO) algorithm. Meanwhile, an energy consumption modeling framework is introduced to quantitatively compare the theoretical energy consumption between SNNs and conventional Artificial Neural Networks (ANNs). A dynamic two-stage reward adjustment mechanism and optimized observation space further improve learning efficiency and policy accuracy. Experiments on the Isaac Gym simulation platform demonstrate that the proposed method achieves superior performance under realistic physical constraints. Comparative evaluations with conventional PPO and ANN baselines validate the scalability and energy efficiency of the proposed approach in dynamic robotic manipulation tasks.