Parameter Stress Analysis in Reinforcement Learning: Applying Synaptic Filtering to Policy Networks

TL;DR

This paper introduces a framework for analyzing the robustness of reinforcement learning policy parameters by applying synaptic filtering and adversarial attacks, identifying fragile, robust, and antifragile parameters to enhance policy resilience.

Contribution

It presents a novel method combining internal and external stress tests to classify parameters and improve RL policy robustness using synaptic filtering techniques.

Findings

Identification of antifragile parameters that improve policy performance under stress

Validation of the framework on PPO agents in Mujoco environments

Demonstration of targeted filtering to enhance RL policy robustness

Abstract

This paper explores reinforcement learning (RL) policy robustness by systematically analyzing network parameters under internal and external stresses. \textcolor{black}{We apply synaptic filtering methods using high-pass, low-pass, and pulse-wave filters from} \citep{pravin2024fragility}, as an internal stress by selectively perturbing parameters, while adversarial attacks apply external stress through modified agent observations. This dual approach enables the classification of parameters as \textit{fragile}, \textit{robust}, or \textit{antifragile}, based on their influence on policy performance in clean and adversarial settings. Parameter scores are defined to quantify these characteristics, and the framework is validated on proximal policy optimization (PPO)-trained agents in Mujoco continuous control environments. The results highlight the presence of antifragile parameters that enhance policy performance under stress, demonstrating the potential of targeted filtering techniques to improve RL policy adaptability. These insights provide a foundation for future advancements in the design of robust and antifragile RL systems.