Implicit differentiation with second-order derivatives and benchmarks in finite-element-based differentiable physics

TL;DR

This paper develops a framework for computing second-order derivatives (Hessians) in finite-element-based differentiable physics, enabling improved optimization in PDE-constrained problems.

Contribution

It introduces a novel method for implicit Hessian computation using automatic differentiation tools, validated through benchmarks across various physics problems.

Findings

Exact Hessians accelerate convergence in nonlinear inverse problems.

L-BFGS-B performs well for linear cases without second-order derivatives.

Second-order information enhances the robustness of differentiable physics engines.

Abstract

Differentiable programming is revolutionizing computational science by enabling automatic differentiation (AD) of numerical simulations. While first-order gradients are well-established, second-order derivatives (Hessians) for implicit functions in finite-element-based differentiable physics remain underexplored. This work bridges this gap by deriving and implementing a framework for implicit Hessian computation in PDE-constrained optimization problems. We leverage primitive AD tools (Jacobian-vector product/vector-Jacobian product) to build an algorithm for Hessian-vector products and validate the accuracy against finite difference approximations. Four benchmarks spanning linear/nonlinear, 2D/3D, and single/coupled-variable problems demonstrate the utility of second-order information. Results show that the Newton-CG method with exact Hessians accelerates convergence for nonlinear inverse problems (e.g., traction force identification, shape optimization), while the L-BFGS-B method suffices for linear cases. Our work provides a robust foundation for integrating second-order implicit differentiation into differentiable physics engines, enabling faster and more reliable optimization.