Differentiable Singular Value Decomposition

TL;DR

This paper introduces two efficient algorithms for computing derivatives of singular value decomposition (SVD) for complex matrices, enabling scalable large-scale optimization in modal analysis.

Contribution

It presents novel adjoint and RAD-based algorithms for accurate, scalable SVD derivatives, outperforming existing methods in precision and efficiency.

Findings

Matched FD results to 5-7 digits for accuracy

Demonstrated scalability on large turbulence dataset

Enabled large-scale design optimization using SVD derivatives

Abstract

Singular value decomposition is widely used in modal analysis, such as proper orthogonal decomposition and resolvent analysis, to extract key features from complex problems. SVD derivatives need to be computed efficiently to enable the large scale design optimization. However, for a general complex matrix, no method can accurately compute this derivative to machine precision and remain scalable with respect to the number of design variables without requiring the all of the singular variables. We propose two algorithms to efficiently compute this derivative based on the adjoint method and reverse automatic differentiation and RAD-based singular value derivative formula. Differentiation results for each method proposed were compared with FD results for one square and one tall rectangular matrix example and matched with the FD results to about 5 to 7 digits. Finally, we demonstrate the scalability of the proposed method by calculating the derivatives of singular values with respect to the snapshot matrix derived from the POD of a large dataset for a laminar-turbulent transitional flow over a flat plate, sourced from the John Hopkins turbulence database.