Edge AIBench: Towards Comprehensive End-to-end Edge Computing Benchmarking

TL;DR

Edge AIBench provides a comprehensive end-to-end benchmarking framework for edge computing, modeling key application scenarios across all layers to evaluate performance, privacy, and security in a unified manner.

Contribution

It introduces a new end-to-end benchmarking suite for edge computing that considers all three layers and models four typical application scenarios.

Findings

Models four key edge applications: ICU monitor, surveillance, smart home, autonomous vehicle

Includes a federated learning framework for privacy and security evaluation

Part of an open-source project for community use

Abstract

In edge computing scenarios, the distribution of data and collaboration of workloads on different layers are serious concerns for performance, privacy, and security issues. So for edge computing benchmarking, we must take an end-to-end view, considering all three layers: client-side devices, edge computing layer, and cloud servers. Unfortunately, the previous work ignores this most important point. This paper presents the BenchCouncil's coordinated e ort on edge AI benchmarks, named Edge AIBench. In total, Edge AIBench models four typical application scenarios: ICU Patient Monitor, Surveillance Camera, Smart Home, and Autonomous Vehicle with the focus on data distribution and workload collaboration on three layers. Edge AIBench is a part of the open-source AIBench project, publicly available from http://www.benchcouncil.org/AIBench/index.html. We also build an edge computing testbed with a federated learning framework to resolve performance, privacy, and security issues.