Parameterized Complexity of Stationarity Testing for Piecewise-Affine Functions and Shallow CNN Losses

TL;DR

This paper investigates the parameterized complexity of testing approximate stationarity for piecewise-affine functions and shallow ReLU CNN losses, revealing fixed-dimensional tractability and hardness results.

Contribution

It provides XP algorithms for tractable cases, proves W[1]-hardness for others, and establishes lower bounds under ETH, extending results to shallow CNN training losses.

Findings

XP algorithms in fixed dimension for tractable cases

W[1]-hardness for the intractable sides

Lower bounds ruling out certain algorithmic running times

Abstract

We study the parameterized complexity of testing approximate first-order stationarity at a prescribed point for continuous piecewise-affine (PA) functions, a basic task in nonsmooth optimization. PA functions form a canonical model for nonsmooth stationarity testing and capture the local polyhedral geometry that appears in ReLU-type training losses. Recent work by Tian and So (SODA 2025) shows that testing approximate stationarity notions for PA functions is computationally intractable in the worst case, and identifies fixed-dimensional tractability as an open direction. We address this direction from the viewpoint of parameterized complexity, with the ambient dimension $d$ as the parameter. In this paper, we give XP algorithms in fixed dimension for the tractable sides, and prove W[1]-hardness for the complementary sides. Moreover, lower bounds under the Exponential Time Hypothesis rule out algorithms running in time $\rho(d)\size^{o(d)}$ for any computable function $\rho$, where $\size$ denotes the total binary encoding length of the stationarity-testing instance. As a further consequence, our results yield the corresponding parameterized complexity picture for testing local minimality of continuous PA functions. We further extend our hardness results to a family of shallow ReLU CNN training losses, with stationarity tested in the trainable weight space. Thus, the same parameterized-complexity picture also appears for simple CNN training losses.