Testing Approximate Stationarity Concepts for Piecewise Affine Functions

TL;DR

This paper investigates the computational complexity of detecting approximate stationary points in piecewise affine functions, introduces efficient algorithms, and provides new theoretical insights into subdifferential calculus.

Contribution

It establishes the intractability of first-order stationarity testing, proposes a polynomial-time relaxation, and introduces the first oracle-polynomial-time algorithm for near-stationarity detection.

Findings

Testing first-order stationarity is NP-hard.

A tight characterization of the convex subdifferential sum rule.

An efficient algorithm for detecting near-approximate stationary points.

Abstract

We study the basic computational problem of detecting approximate stationary points for continuous piecewise affine (PA) functions. Our contributions span multiple aspects, including complexity, regularity, and algorithms. Specifically, we show that testing first-order approximate stationarity concepts, as defined by commonly used generalized subdifferentials, is computationally intractable unless P=NP. To facilitate computability, we consider a polynomial-time solvable relaxation by abusing the convex subdifferential sum rule and establish a tight characterization of its exactness. Furthermore, addressing an open issue motivated by the need to terminate the subgradient method in finite time, we introduce the first oracle-polynomial-time algorithm to detect so-called near-approximate stationary points for PA functions. A notable byproduct of our development in regularity is the first necessary and sufficient condition for the validity of an equality-type (Clarke) subdifferential sum rule. Our techniques revolve around two new geometric notions for convex polytopes and may be of independent interest in nonsmooth analysis. Moreover, some corollaries of our work on complexity and algorithms for stationarity testing address open questions in the literature. To demonstrate the versatility of our results, we complement our findings with applications to a series of structured piecewise smooth functions, including $\rho$-margin-loss SVM, piecewise affine regression, and nonsmooth neural networks.