Enhanced-FQL($\lambda$), an Efficient and Interpretable RL with novel Fuzzy Eligibility Traces and Segmented Experience Replay

TL;DR

Enhanced-FQL(λ) is a fuzzy reinforcement learning framework that combines novel fuzzy eligibility traces and segmented experience replay to improve sample efficiency and interpretability in continuous control tasks.

Contribution

The paper introduces a new fuzzy RL method with fuzzy eligibility traces and segmented experience replay, offering a stable, interpretable, and sample-efficient alternative to neural network-based approaches.

Findings

Improves sample efficiency and reduces variance on Cart-Pole benchmark.

Maintains competitive performance with DDPG baseline.

Proven convergence under standard theoretical assumptions.

Abstract

This paper introduces a fuzzy reinforcement learning framework, Enhanced-FQL($\lambda$), that integrates novel Fuzzified Eligibility Traces (FET) and Segmented Experience Replay (SER) into fuzzy Q-learning with the Fuzzified Bellman Equation (FBE) for continuous control. The proposed approach employs an interpretable fuzzy rule base instead of complex neural architectures, while maintaining competitive performance through two key innovations: a fuzzified Bellman equation with eligibility traces for stable multi-step credit assignment, and a memory-efficient segment-based experience replay mechanism for enhanced sample efficiency. Theoretical analysis proves the proposed method convergence under standard assumptions. On the Cart--Pole benchmark, Enhanced-FQL($\lambda$) improves sample efficiency and reduces variance relative to $n$-step fuzzy TD and fuzzy SARSA($\lambda$), while remaining competitive with the tested DDPG baseline. These results support the proposed framework as an interpretable and computationally compact alternative for moderate-scale continuous control problems.