Noisy Spiking Actor Network for Exploration

TL;DR

This paper introduces NoisySAN, a novel spiking neural network-based exploration method for deep reinforcement learning that employs time-correlated noise and a noise reduction technique to enhance exploration efficiency and stability.

Contribution

The paper presents a new noisy spiking actor network with time-correlated noise and a noise reduction method, improving exploration in deep RL over existing approaches.

Findings

Outperforms state-of-the-art methods on continuous control tasks

Demonstrates robustness to noise in exploration

Achieves stable policy learning with the proposed noise reduction

Abstract

As a general method for exploration in deep reinforcement learning (RL), NoisyNet can produce problem-specific exploration strategies. Spiking neural networks (SNNs), due to their binary firing mechanism, have strong robustness to noise, making it difficult to realize efficient exploration with local disturbances. To solve this exploration problem, we propose a noisy spiking actor network (NoisySAN) that introduces time-correlated noise during charging and transmission. Moreover, a noise reduction method is proposed to find a stable policy for the agent. Extensive experimental results demonstrate that our method outperforms the state-of-the-art performance on a wide range of continuous control tasks from OpenAI gym.