supDQN: Supervised Rewarding Strategy Driven Deep Q-Network for sEMG Signal Decontamination

TL;DR

This paper introduces supDQN, a deep reinforcement learning approach that dynamically selects filtering strategies to decontaminate noisy surface electromyography signals, outperforming conventional methods across various noise levels.

Contribution

It presents a novel supervised reward-driven deep Q-network for adaptive filtering of sEMG signals, improving noise removal efficiency over traditional filters.

Findings

supDQN effectively filters signals at SNRs between -5 dB and +1 dB.

It achieves a lower average nRMSE of 1.1974 compared to conventional filters.

The method adapts filter selection based on noise level, enhancing signal quality.

Abstract

The presence of muscles throughout the active parts of the body such as the upper and lower limbs, makes electromyography-based human-machine interaction prevalent. However, muscle signals are stochastic and noisy. These noises can be regular and irregular. Irregular noises due to movements or electrical switching require dynamic filtering. Conventionally, filters are stacked, which trims and delays the signal unnecessarily. This study introduces a decontamination technique involving a supervised rewarding strategy to drive a deep Q-network-based agent (supDQN). It applies one of three filters to decontaminate a 1sec long surface electromyography signal, which is dynamically contaminated. A machine learning agent identifies whether the signal after filtering is clean or noisy. Accordingly, a reward is generated. The identification accuracy is enhanced by using a local interpretable model-agnostic explanation. The deep Q-network is guided by this reward to select filter optimally while decontaminating a signal. The proposed filtering strategy is tested on four noise levels (-5 dB, -1 dB, +1 dB, +5 dB). supDQN filters the signal desirably when the signal-to-noise ratio (SNR) is between -5 dB to +1 dB. It filters less desirably at high SNR (+5 dB). A normalized root mean square (nRMSE) is formulated to depict the difference of filtered signal from ground truth. This is used to compare supDQN and conventional methods including wavelet denoising with debauchies and symlet wavelet, high order low pass filter, notch filter, and high pass filter. The proposed filtering strategy gives an average value nRMSE of 1.1974, which is lower than the conventional filters.