ARISE: Adaptive Reinforcement Integrated with Swarm Exploration

TL;DR

ARISE is a lightweight, adaptable exploration framework that enhances reinforcement learning by integrating swarm-based particle proposals, significantly improving performance on complex tasks and robustness under reward shifts.

Contribution

ARISE introduces a novel, architecture-agnostic exploration layer combining swarm-based particles with adaptive modulation, improving RL performance and robustness.

Findings

Substantial gains on challenging benchmarks (+46% LunarLander-v3, +22% Hopper-v4)

Marked robustness improvements under non-stationary rewards (+75 points on CartPole)

Ablation confirms the importance of swarm and adaptive components

Abstract

Effective exploration remains a key challenge in RL, especially with non-stationary rewards or high-dimensional policies. We introduce ARISE, a lightweight framework that enhances reinforcement learning by augmenting standard policy-gradient methods with a compact swarm-based exploration layer. ARISE blends policy actions with particle-driven proposals, where each particle represents a candidate policy trajectory sampled in the action space, and modulates exploration adaptively using reward-variance cues. While easy benchmarks exhibit only slight improvements (e.g., +0.7% on CartPole-v1), ARISE yields substantial gains on more challenging tasks, including +46% on LunarLander-v3 and +22% on Hopper-v4, while preserving stability on Walker2d and Ant. Under non-stationary reward shifts, ARISE provides marked robustness advantages, outperforming PPO by +75 points on CartPole and improving LunarLander accordingly. Ablation studies confirm that both the swarm component and the adaptive mechanism contribute to the performance. Overall, ARISE offers a simple, architecture-agnostic route to more exploratory and resilient RL agents without altering core algorithmic structures.