A mathematical formalization of data parallel operations
Victor Eijkhout
TL;DR
This paper provides a formal mathematical framework for generalized data parallel operations, enabling automatic derivation of communication and dependency aspects in scientific computing tasks.
Contribution
It introduces a formalization that unifies various data parallel operations and automates the derivation of related computational details.
Findings
Formalization covers common scientific kernels like matrix-vector multiplication.
Automates derivation of MPI messages and task dependencies.
Enhances understanding of data parallel operations through mathematical modeling.
Abstract
We give a mathematical formalization of `generalized data parallel' operations, a concept that covers such common scientific kernels as matrix-vector multiplication, multi-grid coarsening, load distribution, and many more. We show that from a compact specification such computational aspects as MPI messages or task dependencies can be automatically derived.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsDistributed and Parallel Computing Systems · Parallel Computing and Optimization Techniques · Interconnection Networks and Systems