Fuzzy Logic Theory-based Adaptive Reward Shaping for Robust Reinforcement Learning (FARS)

TL;DR

This paper introduces a fuzzy logic-based reward shaping method for reinforcement learning that incorporates human expertise to improve stability, convergence speed, and performance in complex navigation tasks.

Contribution

It proposes a novel fuzzy logic-based reward shaping approach that encodes expert knowledge for more stable and adaptive reinforcement learning.

Findings

Fuzzy reward shaping leads to faster convergence in autonomous drone racing.

The method improves success rates by up to 5% in challenging environments.

It reduces variability in performance across different training runs.

Abstract

Reinforcement learning (RL) often struggles in real-world tasks with high-dimensional state spaces and long horizons, where sparse or fixed rewards severely slow down exploration and cause agents to get trapped in local optima. This paper presents a fuzzy logic based reward shaping method that integrates human intuition into RL reward design. By encoding expert knowledge into adaptive and interpreable terms, fuzzy rules promote stable learning and reduce sensitivity to hyperparameters. The proposed method leverages these properties to adapt reward contributions based on the agent state, enabling smoother transitions between fast motion and precise control in challenging navigation tasks. Extensive simulation results on autonomous drone racing benchmarks show stable learning behavior and consistent task performance across scenarios of increasing difficulty. The proposed method achieves faster convergence and reduced performance variability across training seeds in more challenging environments, with success rates improving by up to approximately 5 percent compared to non fuzzy reward formulations.