Neural Network Compression for Reinforcement Learning Tasks

TL;DR

This paper systematically investigates neural network compression techniques like sparsity and pruning in reinforcement learning, achieving up to 400-fold size reduction to enhance energy efficiency and reduce latency across various RL algorithms and environments.

Contribution

It provides a comprehensive analysis of compression methods applied to RL, demonstrating significant size reductions and efficiency improvements.

Findings

Up to 400-fold neural network size reduction.

Effective compression across multiple RL algorithms.

Improved energy and latency efficiency in RL applications.

Abstract

In real applications of Reinforcement Learning (RL), such as robotics, low latency and energy efficient inference is very desired. The use of sparsity and pruning for optimizing Neural Network inference, and particularly to improve energy and latency efficiency, is a standard technique. In this work, we perform a systematic investigation of applying these optimization techniques for different RL algorithms in different RL environments, yielding up to a 400-fold reduction in the size of neural networks.