SwiftRL: Towards Efficient Reinforcement Learning on Real Processing-In-Memory Systems

TL;DR

SwiftRL leverages Processing-In-Memory architectures to significantly accelerate reinforcement learning training, demonstrating near-linear performance scaling and superior results over traditional CPU and GPU systems.

Contribution

The paper introduces a novel approach to accelerate RL training by implementing algorithms on PIM hardware, achieving near-linear scaling and improved performance.

Findings

Superior performance over CPU and GPU implementations

Near-linear scaling achieved with PIM hardware

Effective acceleration of RL algorithms like Q-learning and SARSA

Abstract

Reinforcement Learning (RL) trains agents to learn optimal behavior by maximizing reward signals from experience datasets. However, RL training often faces memory limitations, leading to execution latencies and prolonged training times. To overcome this, SwiftRL explores Processing-In-Memory (PIM) architectures to accelerate RL workloads. We achieve near-linear performance scaling by implementing RL algorithms like Tabular Q-learning and SARSA on UPMEM PIM systems and optimizing for hardware. Our experiments on OpenAI GYM environments using UPMEM hardware demonstrate superior performance compared to CPU and GPU implementations.