funcX: Federated Function as a Service for Science
Zhuozhao Li, Ryan Chard, Yadu Babuji, Ben Galewsky, Tyler, Skluzacek, Kirill Nagaitsev, Anna Woodard, Ben Blaiszik, Josh, Bryan, Daniel S. Katz, Ian Foster, Kyle Chard

TL;DR
funcX is a federated FaaS platform enabling scalable, high-performance remote function execution across diverse endpoints, with efficient container management and data handling, demonstrated on supercomputers with significant performance improvements.
Contribution
This paper introduces funcX, a novel federated FaaS system that decouples management from execution, supporting diverse endpoints and optimizing performance through container strategies and data integration.
Findings
Scales to over 130,000 concurrent workers.
Container warming-aware routing reduces function completion time by up to 61%.
In-memory data store speeds data transfers by up to 3x.
Abstract
funcX is a distributed function as a service (FaaS) platform that enables flexible, scalable, and high performance remote function execution. Unlike centralized FaaS systems, funcX decouples the cloud-hosted management functionality from the edge-hosted execution functionality. funcX's endpoint software can be deployed, by users or administrators, on arbitrary laptops, clouds, clusters, and supercomputers, in effect turning them into function serving systems. funcX's cloud-hosted service provides a single location for registering, sharing, and managing both functions and endpoints. It allows for transparent, secure, and reliable function execution across the federated ecosystem of endpoints--enabling users to route functions to endpoints based on specific needs. funcX uses containers (e.g., Docker, Singularity, and Shifter) to provide common execution environments across endpoints.…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsCloud Computing and Resource Management · Scientific Computing and Data Management · Distributed systems and fault tolerance
