FedAD-Bench: A Unified Benchmark for Federated Unsupervised Anomaly Detection in Tabular Data

TL;DR

FedAD-Bench introduces a comprehensive benchmark for evaluating unsupervised anomaly detection methods in federated learning, addressing the lack of standardized evaluation and revealing unique challenges and benefits of FL in anomaly detection.

Contribution

This work presents the first unified benchmark for federated unsupervised anomaly detection, systematically comparing recent models and analyzing FL-specific challenges and advantages.

Findings

Federated learning can outperform centralized methods in certain scenarios due to reduced overfitting.

Model aggregation inefficiencies are a key challenge in federated anomaly detection.

The benchmark promotes reproducibility and fair comparison in future research.

Abstract

The emergence of federated learning (FL) presents a promising approach to leverage decentralized data while preserving privacy. Furthermore, the combination of FL and anomaly detection is particularly compelling because it allows for detecting rare and critical anomalies (usually also rare in locally gathered data) in sensitive data from multiple sources, such as cybersecurity and healthcare. However, benchmarking the performance of anomaly detection methods in FL environments remains an underexplored area. This paper introduces FedAD-Bench, a unified benchmark for evaluating unsupervised anomaly detection algorithms within the context of FL. We systematically analyze and compare the performance of recent deep learning anomaly detection models under federated settings, which were typically assessed solely in centralized settings. FedAD-Bench encompasses diverse datasets and metrics to provide a holistic evaluation. Through extensive experiments, we identify key challenges such as model aggregation inefficiencies and metric unreliability. We present insights into FL's regularization effects, revealing scenarios in which it outperforms centralized approaches due to its inherent ability to mitigate overfitting. Our work aims to establish a standardized benchmark to guide future research and development in federated anomaly detection, promoting reproducibility and fair comparison across studies.