Twice Epi-Differentiability of Spectral Functions and its applications

TL;DR

This paper characterizes twice epi-differentiability of spectral functions without relying on convexity, enabling new insights into eigenvalue functions and applications in statistics and robust PCA.

Contribution

It provides a novel characterization of twice epi-differentiability for spectral functions that relaxes convexity assumptions and applies to eigenvalue-based regularization.

Findings

Characterization of twice epi-differentiability via spectral representation

Application to proto-differentiability of subgradient mappings

Analysis of eigenvalue functions in regularization contexts

Abstract

Second-order variational properties have been shown to play important theoretical and numerical roles for different classes of optimization problems. Among such properties, twice epi-differentiability has a special place because of its ubiquitous presence in various classes of extended-real-valued functions that are important for optimization problems. We provide a useful characterization of this property for spectral functions by demonstrating that it can be characterized via the same property of the symmetric part of the spectral representation of an eigenvalue function. Our approach allows us to bypass the rather restrictive convexity assumption, used in many recent works that targeted second-order variational properties of spectral functions. By this theoretical tool, several applications on the proto-differentiability of subgradient mappings, the directional differentiability of the proximal mapping of spectral functions are achieved. We finally use our established theory to study twice epi-differentiability of leading eigenvalue functions and practical regularization terms that have important applications in statistics and the robust PCA.