Porting the Nonlinear Optimization Library HiOp to Accelerator-Based Hardware Architectures

TL;DR

This paper introduces a novel interior point method implementation in the HiOp library that runs entirely on hardware accelerators by compressing sparse problems into dense ones, enabling efficient GPU utilization.

Contribution

It presents a new formulation of interior point methods that avoids sparse linear solvers, facilitating implementation on hardware accelerators like GPUs.

Findings

Feasibility demonstrated for GPU-based interior point methods.

Achieved over 90% GPU processing power utilization.

Provides a baseline for future hardware-accelerated nonlinear optimization.

Abstract

While interior point methods have been the centerpiece of nonlinear programming tools used in science and engineering, their reliance on linear solvers that can tackle sparse symmetric indefinite and highly ill-conditioned problems made it difficult to implement them effectively on hardware accelerators. At this time, there are few sparse linear solvers that can be used in this context. Here, we present a novel formulation of an interior point method implemented in our HiOp library, which is designed to be able to run entirely on hardware accelerators. This formulation avoids dependence on sparse solvers altogether, which is achieved by compressing the underlying sparse linear problem into a dense one of manageable size. We demonstrate feasibility of this approach and provide a baseline for future interior point method implementations on hardware accelerators. Our investigation is motivated by problems arising in optimal power flow analysis in power systems engineering and our approach is tailored to the broad class of problems arising in that important domain. We also demonstrate utility of modern programming models based on performance portability libraries, namely, Umpire and RAJA. We discuss trade-offs between performance, portability and development cost in the solution space for this non-linear optimization problem. As a result of this research, we demonstrate for the first time that interior point methods for sparse problems can be efficiently realized on modern computing systems where more than 90% of processing power is in GPUs.