MultiCloud Resource Management using Apache Mesos for Planned Integration with Apache Airavata
Pankaj Saha, Madhusudhan Govindaraju, Suresh Marru, Marlon Pierce

TL;DR
This paper explores integrating Apache Mesos with Apache Airavata to enable multi-cloud resource management and scheduling, including handling VMs without public IPs, to improve job distribution across clouds.
Contribution
It proposes a novel meta-scheduler design using Apache Mesos for multi-cloud environments, addressing resource visibility and job scheduling challenges without requiring public IPs.
Findings
Initial results demonstrate resource management across multiple clouds.
The approach enables scheduling of jobs on VMs without public IPs.
Next steps include developing and testing the meta-scheduler.
Abstract
We discuss initial results and our planned approach for incorporating Apache Mesos based resource management that will enable design and development of scheduling strategies for Apache Airavata jobs so that they can be launched on multiple clouds, wherein several VMs do not have Public IP addresses. We present initial work and next steps on the design of a meta-scheduler using Apache Mesos. Apache Mesos presents a unified view of resources available across several clouds and clusters. Our meta-scheduler can potentially examine and identify the cases where multiple small jobs have been submitted by the same scientists and then redirect job from the same community account or user to different clusters. Our approach uses a NAT firewall to make nodes/VMs, without a Public IP, visible to Mesos for the unified view.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsCloud Computing and Resource Management · Software-Defined Networks and 5G · IoT and Edge/Fog Computing
