Robustness Verification of Polynomial Neural Networks

TL;DR

This paper investigates the robustness verification of polynomial neural networks using algebraic geometry, introducing new measures and methods to analyze decision boundaries and certify robustness.

Contribution

It introduces the use of Euclidean distance degree and discriminants for analyzing robustness, deriving formulas for network architectures, and developing symbolic and homotopy methods for certification.

Findings

ED degree varies with network architecture

Homotopy methods enable exact robustness certification

Lightning self-attention modules have smaller ED degree than generic cubic hypersurfaces

Abstract

We study robustness verification of neural networks via metric algebraic geometry. For polynomial neural networks, certifying a robustness radius amounts to computing the distance to the algebraic decision boundary. We use the Euclidean distance (ED) degree as an intrinsic measure of the complexity of this problem, analyze the associated ED discriminant, and introduce a parameter discriminant that detects parameter values at which the ED degree drops. We derive formulas for the ED degree for several network architectures and characterize the expected number of real critical points in the infinite-width limit. We develop symbolic elimination methods to compute these quantities and homotopy-continuation methods for exact robustness certification. Finally, experiments on lightning self-attention modules reveal decision boundaries with strictly smaller ED degree than generic cubic hypersurfaces of the same ambient dimension.