SAGIPS: A Scalable Asynchronous Generative Inverse Problem Solver

TL;DR

SAGIPS introduces a scalable, asynchronous deep learning workflow for inverse problem solving that efficiently utilizes high-performance computing systems, enabling near-linear scaling and high-quality convergence.

Contribution

The paper presents SAGIPS, a novel asynchronous parallelization method for inverse problem solving with GANs on HPC systems, improving scalability and efficiency.

Findings

Near linear weak scaling demonstrated in experiments

Convergence quality comparable to traditional methods

Effective multi-GPU utilization for complex inverse problems

Abstract

Large scale, inverse problem solving deep learning algorithms have become an essential part of modern research and industrial applications. The complexity of the underlying inverse problem often poses challenges to the algorithm and requires the proper utilization of high-performance computing systems. Most deep learning algorithms require, due to their design, custom parallelization techniques in order to be resource efficient while showing a reasonable convergence. In this paper we introduces a \underline{S}calable \underline{A}synchronous \underline{G}enerative workflow for solving \underline{I}nverse \underline{P}roblems \underline{S}olver (SAGIPS) on high-performance computing systems. We present a workflow that utilizes a parallelization approach where the gradients of the generator network are updated in an asynchronous ring-all-reduce fashion. Experiments with a scientific proxy application demonstrate that SAGIPS shows near linear weak scaling, together with a convergence quality that is comparable to traditional methods. The approach presented here allows leveraging GANs across multiple GPUs, promising advancements in solving complex inverse problems at scale.