Neural Network Optimization for Reinforcement Learning Tasks Using Sparse Computations

TL;DR

This paper introduces a sparse computation method combining pruning and data correlation to optimize neural networks for reinforcement learning, achieving significant reduction in multiplications with minimal or improved performance.

Contribution

It presents a novel sparse computation approach that updates neuron states selectively based on thresholds, reducing computations in RL neural networks.

Findings

Achieved 20-150x reduction in multiplications

No significant performance loss observed

Sometimes performance improved

Abstract

This article proposes a sparse computation-based method for optimizing neural networks for reinforcement learning (RL) tasks. This method combines two ideas: neural network pruning and taking into account input data correlations; it makes it possible to update neuron states only when changes in them exceed a certain threshold. It significantly reduces the number of multiplications when running neural networks. We tested different RL tasks and achieved 20-150x reduction in the number of multiplications. There were no substantial performance losses; sometimes the performance even improved.