Real-World Image Datasets for Federated Learning

TL;DR

This paper introduces a real-world image dataset for federated learning, addressing the lack of authentic data and providing benchmarks for model performance, efficiency, and communication in realistic scenarios.

Contribution

It provides a new high-quality, real-world dataset with detailed annotations and benchmarks for federated learning, which was previously reliant on artificial datasets.

Findings

The dataset contains over 900 images from street cameras with 7 object categories.

Benchmark results for YOLO and Faster R-CNN demonstrate their performance in federated settings.

The dataset reflects non-IID and unbalanced data distributions typical of real-world federated learning.

Abstract

Federated learning is a new machine learning paradigm which allows data parties to build machine learning models collaboratively while keeping their data secure and private. While research efforts on federated learning have been growing tremendously in the past two years, most existing works still depend on pre-existing public datasets and artificial partitions to simulate data federations due to the lack of high-quality labeled data generated from real-world edge applications. Consequently, advances on benchmark and model evaluations for federated learning have been lagging behind. In this paper, we introduce a real-world image dataset. The dataset contains more than 900 images generated from 26 street cameras and 7 object categories annotated with detailed bounding box. The data distribution is non-IID and unbalanced, reflecting the characteristic real-world federated learning scenarios. Based on this dataset, we implemented two mainstream object detection algorithms (YOLO and Faster R-CNN) and provided an extensive benchmark on model performance, efficiency, and communication in a federated learning setting. Both the dataset and algorithms are made publicly available.