Multimodal Spiking Neural Network for Space Robotic Manipulation

TL;DR

This paper introduces a multimodal spiking neural network framework for space robotic manipulation, combining geometric, tactile, and semantic data, enhanced by curriculum reinforcement learning, to improve autonomy and efficiency in space operations.

Contribution

The paper presents a novel multimodal control framework using SNNs with a curriculum reinforcement learning scheme for space robotic arms, addressing resource constraints and environmental awareness.

Findings

Outperforms baseline methods in success rate

Demonstrates high energy efficiency

Effective across multiple manipulation tasks

Abstract

This paper presents a multimodal control framework based on spiking neural networks (SNNs) for robotic arms aboard space stations. It is designed to cope with the constraints of limited onboard resources while enabling autonomous manipulation and material transfer in space operations. By combining geometric states with tactile and semantic information, the framework strengthens environmental awareness and contributes to more robust control strategies. To guide the learning process progressively, a dual-channel, three-stage curriculum reinforcement learning (CRL) scheme is further integrated into the system. The framework was tested across a range of tasks including target approach, object grasping, and stable lifting with wall-mounted robotic arms, demonstrating reliable performance throughout. Experimental evaluations demonstrate that the proposed method consistently outperforms baseline approaches in both task success rate and energy efficiency. These findings highlight its suitability for real-world aerospace applications.