Batched Second-Order Adjoint Sensitivity for Reduced Space Methods

TL;DR

This paper introduces a GPU-optimized batched automatic differentiation method for efficiently computing second-order sensitivities and reduced Hessians in large-scale implicit nonlinear systems, significantly accelerating power network optimization tasks.

Contribution

The paper develops a novel batched AutoDiff backend and a parallel adjoint-adjoint algorithm tailored for GPUs to compute second-order sensitivities in reduced space problems.

Findings

GPU implementation is 30 times faster than CPU for large instances.

Method effectively computes reduced Hessians for power network balance equations.

Parallel AutoDiff enhances efficiency of second-order sensitivity analysis.

Abstract

This paper presents an efficient method for extracting the second-order sensitivities from a system of implicit nonlinear equations on upcoming graphical processing units (GPU) dominated computer systems. We design a custom automatic differentiation (AutoDiff) backend that targets highly parallel architectures by extracting the second-order information in batch. When the nonlinear equations are associated to a reduced space optimization problem, we leverage the parallel reverse-mode accumulation in a batched adjoint-adjoint algorithm to compute efficiently the reduced Hessian of the problem. We apply the method to extract the reduced Hessian associated to the balance equations of a power network, and show on the largest instances that a parallel GPU implementation is 30 times faster than a sequential CPU reference based on UMFPACK.