The Impact of Quantization and Pruning on Deep Reinforcement Learning Models

TL;DR

This paper investigates how quantization and pruning affect deep reinforcement learning models, focusing on performance trade-offs, energy efficiency, and deployment in resource-limited environments.

Contribution

It provides a comprehensive analysis of the effects of quantization and pruning on DRL models' performance, energy use, and resource requirements, offering deployment guidelines.

Findings

Model size decreases with compression techniques.

Energy efficiency does not significantly improve despite size reduction.

Performance trade-offs depend on the specific compression method and environment.

Abstract

Deep reinforcement learning (DRL) has achieved remarkable success across various domains, such as video games, robotics, and, recently, large language models. However, the computational costs and memory requirements of DRL models often limit their deployment in resource-constrained environments. The challenge underscores the urgent need to explore neural network compression methods to make RDL models more practical and broadly applicable. Our study investigates the impact of two prominent compression methods, quantization and pruning on DRL models. We examine how these techniques influence four performance factors: average return, memory, inference time, and battery utilization across various DRL algorithms and environments. Despite the decrease in model size, we identify that these compression techniques generally do not improve the energy efficiency of DRL models, but the model size decreases. We provide insights into the trade-offs between model compression and DRL performance, offering guidelines for deploying efficient DRL models in resource-constrained settings.