Seemless Utilization of Heterogeneous XSede Resources to Accelerate Processing of a High Value Functional Neuroimaging Dataset

TL;DR

This paper details how leveraging heterogeneous XSede resources on the Open Science Grid significantly accelerated processing of a large neuroimaging dataset, achieving over 15 million core hours with minimal effort and interference.

Contribution

It demonstrates a practical approach to utilizing opportunistic HPC resources across heterogeneous systems to efficiently process large scientific datasets.

Findings

Achieved over 15 million core hours in 7 months.

Boosted computing capacity more than 5-fold.

Minimal software development and no interference with other HPC users.

Abstract

We describe the technical effort used to process a voluminous high value human neuroimaging dataset on the Open Science Grid with opportunistic use of idle HPC resources to boost computing capacity more than 5-fold. With minimal software development effort and no discernable competitive interference with other HPC users, this effort delivered 15,000,000 core hours over 7 months.