Analyzing the HCP Datasets using GPUs: The Anatomy of a Science Engagement

TL;DR

This paper details the transformation of a neuroimaging data processing workflow from a 21-day CPU-based process to a 2-hour GPU-accelerated pipeline, highlighting performance improvements and infrastructure enhancements.

Contribution

It introduces GPU acceleration to the FSL BedpostX workflow, demonstrating significant speedup and providing insights into system bottlenecks and collaborative software deployment.

Findings

GPU acceleration reduced processing time from 21 days to 2 hours.

Network and compute bottlenecks were identified and addressed.

Open licensing and collaboration platforms facilitated software improvements.

Abstract

This paper documents the experience improving the performance of a data processing workflow for analysis of the Human Connectome Project's HCP900 data set. It describes how network and compute bottlenecks were discovered and resolved during the course of a science engagement. A series of computational enhancements to the stock FSL BedpostX workflow are described. These enhancements migrated the workflow from a slow serial execution of computations resulting from Slurm scheduler incompatibilities to eventual execution on GPU resources, going from a 21-day execution on a single CPU core to a 2 hour execution on a GPU. This workflow contributed a vital use-case to the build-out of the campus compute cluster with additional GPUs and resulted in enhancements to network bandwidth. It also shares insights on potential improvements to distribution of scientific software to avoid stagnation in site-specific deployment decisions. The discussion highlights the advantages of open licenses and popular code collaboration sites like GitHub.com in feeding contributions upstream.