Pilot-Edge: Distributed Resource Management Along the Edge-to-Cloud Continuum

TL;DR

Pilot-Edge introduces a unified resource management abstraction for distributed edge-to-cloud systems, enabling efficient deployment and task placement of IoT applications with diverse performance and privacy requirements.

Contribution

It presents a novel abstraction based on the pilot concept, integrating FaaS for resource and workload management across the edge-to-cloud continuum.

Findings

Effective resource management across distributed infrastructures

Supports application-specific task placement decisions

Demonstrated on machine learning workloads with performance benefits

Abstract

Many science and industry IoT applications necessitate data processing across the edge-to-cloud continuum to meet performance, security, cost, and privacy requirements. However, diverse abstractions and infrastructures for managing resources and tasks across the edge-to-cloud scenario are required. We propose Pilot-Edge as a common abstraction for resource management across the edge-to-cloud continuum. Pilot-Edge is based on the pilot abstraction, which decouples resource and workload management, and provides a Function-as-a-Service (FaaS) interface for application-level tasks. The abstraction allows applications to encapsulate common functions in high-level tasks that can then be configured and deployed across the continuum. We characterize Pilot-Edge on geographically distributed infrastructures using machine learning workloads (e.g., k-means and auto-encoders). Our experiments demonstrate how Pilot-Edge manages distributed resources and allows applications to evaluate task placement based on multiple factors (e.g., model complexities, throughput, and latency).