On the Interplay Between Sparsity and Training in Deep Reinforcement Learning

TL;DR

This paper investigates how different sparse neural network architectures influence the performance of deep reinforcement learning in image-based tasks, highlighting the importance of architecture choice and weight training strategies.

Contribution

It provides a comparative analysis of sparse versus dense architectures in deep reinforcement learning, emphasizing the impact of sparsity and weight training on learning outcomes.

Findings

Sparse architectures significantly affect learning performance.

The effectiveness of sparse structures depends on whether weights are fixed or learned.

Choosing the right sparse architecture is crucial for optimal reinforcement learning performance.

Abstract

We study the benefits of different sparse architectures for deep reinforcement learning. In particular, we focus on image-based domains where spatially-biased and fully-connected architectures are common. Using these and several other architectures of equal capacity, we show that sparse structure has a significant effect on learning performance. We also observe that choosing the best sparse architecture for a given domain depends on whether the hidden layer weights are fixed or learned.