Run-time Parameter Sensitivity Analysis Optimizations

TL;DR

This paper presents a distributed computing optimization for parameter sensitivity analysis of pathology image processing, significantly reducing computation time and improving reuse efficiency on large datasets.

Contribution

It introduces a novel computation reuse strategy that enhances efficiency in large-scale parameter sensitivity analysis, even with limited memory.

Findings

Achieved over 92% parallel efficiency on 256 nodes.

Improved reuse strategies by up to 2.8 times.

Enabled faster analysis of large pathology datasets.

Abstract

Efficient execution of parameter sensitivity analysis (SA) is critical to allow for its routinely use. The pathology image processing application investigated in this work processes high-resolution whole-slide cancer tissue images from large datasets to characterize and classify the disease. However, the application is parameterized and changes in parameter values may significantly affect its results. Thus, understanding the impact of parameters to the output using SA is important to draw reliable scientific conclusions. The execution of the application is rather compute intensive, and a SA requires it to process the input data multiple times as parameter values are systematically varied. Optimizing this process is then important to allow for SA to be executed with large datasets. In this work, we employ a distributed computing system with novel computation reuse optimizations to accelerate SA. The new computation reuse strategy can maximize reuse even with limited memory availability where previous approaches would not be able to fully take advantage of reuse. The proposed solution was evaluated on an environment with 256 nodes (7168 CPU-cores) attaining a parallel efficiency of over 92%, and improving the previous reuse strategies in up to 2.8x.