Composite Reward Design in PPO-Driven Adaptive Filtering

TL;DR

This paper introduces a novel reinforcement learning framework using PPO with a composite reward for adaptive filtering, outperforming traditional methods in non-stationary environments and demonstrating real-time capabilities.

Contribution

It presents a new PPO-based adaptive filtering approach with a composite reward, enabling robust, real-time denoising in dynamic environments, surpassing classical filters.

Findings

PPO-based filter outperforms classical filters in various noise conditions.

The method generalizes well beyond training data.

Achieves real-time adaptive filtering performance.

Abstract

Model-free and reinforcement learning-based adaptive filtering methods are gaining traction for denoising in dynamic, non-stationary environments such as wireless signal channels. Traditional filters like LMS, RLS, Wiener, and Kalman are limited by assumptions of stationary or requiring complex fine-tuning or exact noise statistics or fixed models. This letter proposes an adaptive filtering framework using Proximal Policy Optimization (PPO), guided by a composite reward that balances SNR improvement, MSE reduction, and residual smoothness. Experiments on synthetic signals with various noise types show that our PPO agent generalizes beyond its training distribution, achieving real-time performance and outperforming classical filters. This work demonstrates the viability of policy-gradient reinforcement learning for robust, low-latency adaptive signal filtering.