Benchmarking Performance of Deep Learning Model for Material Segmentation on Two HPC Systems

TL;DR

This paper presents a benchmarking tool that uses deep learning models to evaluate the performance of GPU-accelerated HPC systems during material segmentation tasks, providing insights into system efficiency and environmental impacts.

Contribution

It introduces a novel benchmarking approach utilizing a deep learning model converted from Caffe to PyTorch, applied to two HPC systems for performance analysis.

Findings

Vulcanite often has faster model times but is more affected by environmental factors.

Onyx shows consistent model times across benchmarks.

Performance variability is influenced by environmental conditions.

Abstract

Performance Benchmarking of HPC systems is an ongoing effort that seeks to provide information that will allow for increased performance and improve the job schedulers that manage these systems. We develop a benchmarking tool that utilizes machine learning models and gathers performance data on GPU-accelerated nodes while they perform material segmentation analysis. The benchmark uses a ML model that has been converted from Caffe to PyTorch using the MMdnn toolkit and the MINC-2500 dataset. Performance data is gathered on two ERDC DSRC systems, Onyx and Vulcanite. The data reveals that while Vulcanite has faster model times in a large number of benchmarks, and it is also more subject to some environmental factors that can cause performances slower than Onyx. In contrast the model times from Onyx are consistent across benchmarks.