A Grid Service Broker for Scheduling Distributed Data-Oriented Applications on Global Grids

TL;DR

This paper introduces a grid service broker that efficiently manages data and computational resources for distributed data-oriented applications in global grid environments, enhancing e-Science collaborations.

Contribution

It presents a novel grid broker that automates resource discovery, job mapping, deployment, and data access for distributed scientific applications, supporting dynamic parametric programming.

Findings

Successfully deployed high energy physics analysis on a global grid

Enhanced data access and job scheduling efficiency

Demonstrated scalability across international resources

Abstract

The next generation of scientific experiments and studies, popularly called as e-Science, is carried out by large collaborations of researchers distributed around the world engaged in analysis of huge collections of data generated by scientific instruments. Grid computing has emerged as an enabler for e-Science as it permits the creation of virtual organizations that bring together communities with common objectives. Within a community, data collections are stored or replicated on distributed resources to enhance storage capability or efficiency of access. In such an environment, scientists need to have the ability to carry out their studies by transparently accessing distributed data and computational resources. In this paper, we propose and develop a Grid broker that mediates access to distributed resources by (a) discovering suitable data sources for a given analysis scenario, (b) suitable computational resources, (c) optimally mapping analysis jobs to resources, (d) deploying and monitoring job execution on selected resources, (e) accessing data from local or remote data source during job execution and (f) collating and presenting results. The broker supports a declarative and dynamic parametric programming model for creating grid applications. We have used this model in grid-enabling a high energy physics analysis application (Belle Analysis Software Framework). The broker has been used in deploying Belle experiment data analysis jobs on a grid testbed, called Belle Analysis Data Grid, having resources distributed across Australia interconnected through GrangeNet.