Scalable higher-order nonlinear solvers via higher-order automatic differentiation

TL;DR

This paper introduces scalable higher-order nonlinear solvers using Taylor-mode automatic differentiation, demonstrating efficient computation of derivatives and improved convergence for large-scale and ill-conditioned problems.

Contribution

It develops and implements higher-order convergence methods, especially Halley's method, for large-scale nonlinear problems using efficient derivative computations.

Findings

Halley's method outperforms Newton's method on large-scale problems

Efficient higher-order derivatives enable scalable solvers

Method scales well to ill-conditioned and stiff problems

Abstract

This paper demonstrates new methods and implementations of nonlinear solvers with higher-order of convergence, which is achieved by efficiently computing higher-order derivatives. Instead of computing full derivatives, which could be expensive, we compute directional derivatives with Taylor-mode automatic differentiation. We first implement Householder's method with arbitrary order for one variable, and investigate the trade-off between computational cost and convergence order. We find that the second-order variant, i.e., Halley's method, to be the most valuable, and further generalize Halley's method to systems of nonlinear equations and demonstrate that it can scale efficiently to large-scale problems. We further apply Halley's method on solving large-scale ill-conditioned nonlinear problems, as well as solving nonlinear equations inside stiff ODE solvers, and demonstrate that it could outperform Newton's method.