# Noise Resilience of Successor and Predecessor Feature Algorithms in One- and Two-Dimensional Environments

## TL;DR

This paper compares how well different reinforcement learning algorithms handle noise in one- and two-dimensional environments, finding that successor feature algorithms perform better under high noise.

## Contribution

The study introduces a novel comparative analysis showing that successor feature algorithms outperform traditional methods in noisy environments.

## Key findings

- SF algorithms achieve 2216.88±3.83 cumulative rewards under high noise (σ=0.5) in one-dimensional environments.
- In two-dimensional environments, SF performs best at moderate noise (σ=0.25) with 2886.03±1.63 cumulative rewards.
- The λ parameter in PF learning significantly affects performance, with λ=0.7 yielding the best results.

## Abstract

Noisy inputs pose significant challenges for reinforcement learning (RL) agents navigating real-world environments. While animals demonstrate robust spatial learning under dynamic conditions, the mechanisms underlying this resilience remain understudied in RL frameworks. This paper introduces a novel comparative analysis of predecessor feature (PF) and successor feature (SF) algorithms under controlled noise conditions, revealing several insights. Our key innovation lies in demonstrating that SF algorithms achieve superior noise resilience compared to traditional approaches, with cumulative rewards of 2216.88±3.83 (mean ± SEM), even under high noise conditions (σ=0.5) in one-dimensional environments, while Q learning achieves only 19.22±0.57. In two-dimensional environments, we discover an unprecedented nonlinear relationship between noise level and algorithm performance, with SF showing optimal performance at moderate noise levels (σ=0.25), achieving cumulative rewards of 2886.03±1.63 compared to 2798.16±3.54 for Q learning. The λ parameter in PF learning is a significant factor, with λ=0.7 consistently achieving higher λ values under most noise conditions. These findings bridge computational neuroscience and RL, offering practical insights for developing noise-resistant learning systems. Our results have direct applications in robotics, autonomous navigation, and sensor-based AI systems, particularly in environments with inherent observational uncertainty.