Higher-Order Singular-Value Derivatives of Rectangular Real Matrices

TL;DR

This paper develops a theoretical framework using operator perturbation theory to derive higher-order derivatives of singular values in rectangular matrices, enabling advanced spectral sensitivity analysis.

Contribution

It introduces a novel approach by embedding matrices into self-adjoint operators to obtain closed-form higher-order derivatives of singular values, including the Hessian, which was previously unavailable.

Findings

Derived general formulas for n-th order derivatives of singular values.

Obtained explicit Hessian expressions for singular values.

Bridged operator theory with matrix analysis for spectral sensitivity.

Abstract

We present a theoretical framework for deriving the general $n$-th order Fr\'echet derivatives of singular values in real rectangular matrices, by leveraging reduced resolvent operators from Kato's analytic perturbation theory for self-adjoint operators. Deriving closed-form expressions for higher-order derivatives of singular values is notoriously challenging through standard matrix-analysis techniques. To overcome this, we treat a real rectangular matrix as a compact operator on a finite-dimensional Hilbert space, and embed the rectangular matrix into a block self-adjoint operator so that non-symmetric perturbations are captured. Applying Kato's asymptotic eigenvalue expansion to this construction, we obtain a general, closed-form expression for the infinitesimal $n$-th order spectral variations. Specializing to $n=2$ and deploying on a Kronecker-product representation with matrix convention yield the Hessian of a singular value, not found in literature. By bridging abstract operator-theoretic perturbation theory with matrices, our framework equips researchers with a practical toolkit for higher-order spectral sensitivity studies in random matrix applications (e.g., adversarial perturbation in deep learning).