FRL-FI: Transient Fault Analysis for Federated Reinforcement Learning-Based Navigation Systems

TL;DR

This paper evaluates the fault tolerance of federated reinforcement learning navigation systems and proposes two efficient fault detection methods that significantly improve resilience with minimal overhead.

Contribution

It introduces a comprehensive fault analysis for FRL navigation systems and presents two novel, cost-effective fault detection and recovery techniques.

Findings

Fault tolerance varies with fault models and system parameters.

Proposed techniques achieve up to 3.3x resilience improvement.

Overhead of the methods is less than 2.7%.

Abstract

Swarm intelligence is being increasingly deployed in autonomous systems, such as drones and unmanned vehicles. Federated reinforcement learning (FRL), a key swarm intelligence paradigm where agents interact with their own environments and cooperatively learn a consensus policy while preserving privacy, has recently shown potential advantages and gained popularity. However, transient faults are increasing in the hardware system with continuous technology node scaling and can pose threats to FRL systems. Meanwhile, conventional redundancy-based protection methods are challenging to deploy on resource-constrained edge applications. In this paper, we experimentally evaluate the fault tolerance of FRL navigation systems at various scales with respect to fault models, fault locations, learning algorithms, layer types, communication intervals, and data types at both training and inference stages. We further propose two cost-effective fault detection and recovery techniques that can achieve up to 3.3x improvement in resilience with <2.7% overhead in FRL systems.