Extended Abstract: Productive Parallel Programming with Parsl
Kyle Chard, Yadu Babuji, Anna Woodard, Ben Clifford, Zhuozhao Li,, Mihael Hategan, Ian Foster, Mike Wilde, Daniel S. Katz
TL;DR
Parsl is a Python library that simplifies parallel programming by allowing developers to annotate functions for concurrent execution, automatically managing dependencies and resource utilization across diverse computing environments.
Contribution
This work introduces Parsl, a flexible Python library that enables easy specification and execution of parallel programs on various computing systems without code modification.
Findings
Supports execution on laptops to supercomputers
Automatically manages dependencies and resource allocation
Simplifies parallel programming in Python
Abstract
Parsl is a parallel programming library for Python that aims to make it easy to specify parallelism in programs and to realize that parallelism on arbitrary parallel and distributed computing systems. Parsl relies on developers annotating Python functions-wrapping either Python or external applications-to indicate that these functions may be executed concurrently. Developers can then link together functions via the exchange of data. Parsl establishes a dynamic dependency graph and sends tasks for execution on connected resources when dependencies are resolved. Parsl's runtime system enables different compute resources to be used, from laptops to supercomputers, without modification to the Parsl program.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.