Closed loop interactions between spiking neural network and robotic simulators based on MUSIC and ROS

TL;DR

This paper introduces a middleware connecting ROS and MUSIC to enable real-time, closed-loop interactions between neural network simulations and robotic systems, facilitating complex experiments in neurorobotics and neuroscience.

Contribution

It presents a novel middleware solution that efficiently couples robotic and neural simulators for real-time closed-loop experiments.

Findings

Enables real-time closed-loop neural-robot interactions

Supports complex, reproducible stimuli for neural systems

Extends research capabilities in neurorobotics and neuroscience

Abstract

In order to properly assess the function and computational properties of simulated neural systems, it is necessary to account for the nature of the stimuli that drive the system. However, providing stimuli that are rich and yet both reproducible and amenable to experimental manipulations is technically challenging, and even more so if a closed-loop scenario is required. In this work, we present a novel approach to solve this problem, connecting robotics and neural network simulators. We implement a middleware solution that bridges the Robotic Operating System (ROS) to the Multi-Simulator Coordinator (MUSIC). This enables any robotic and neural simulators that implement the corresponding interfaces to be efficiently coupled, allowing real-time performance for a wide range of configurations. This work extends the toolset available for researchers in both neurorobotics and computational neuroscience, and creates the opportunity to perform closed-loop experiments of arbitrary complexity to address questions in multiple areas, including embodiment, agency, and reinforcement learning.