Concave Certificates: Geometric Framework for Distributionally Robust Risk and Complexity Analysis

TL;DR

This paper introduces a geometric framework using concave certificates to provide tight, applicable bounds on distributionally robust risk and complexity, improving over traditional Lipschitz-based methods.

Contribution

It develops a novel concave certificate framework for risk and complexity analysis that is applicable to non-Lipschitz, non-differentiable losses and improves neural network generalization bounds.

Findings

Provides tight bounds on DR risk applicable to complex losses.

Eliminates dependencies on input diameter, network width, and depth in complexity bounds.

Introduces an adversarial score for efficient neural network analysis.

Abstract

Distributionally Robust (DR) optimization aims to certify worst-case risk within a Wasserstein uncertainty set. Current certifications typically rely either on global Lipschitz bounds, which are often conservative, or on local gradient information, which provides only a first-order approximation. This paper introduces a novel geometric framework based on the least concave majorants of the growth rate functions. Our proposed concave certificate establishes a tight bound on DR risk that remains applicable to non-Lipschitz and non-differentiable losses. We extend this framework to complexity analysis, introducing the worst-case generalization bound that complements the standard statistical generalization bound. Furthermore, we utilize this certificate to bound the gap between adversarial and empirical Rademacher complexity, demonstrating that dependencies on input diameter, network width, and depth can be eliminated. For practical application in deep learning, we introduce the adversarial score as a tractable relaxation of the concave certificate that enables efficient and layer-wise analysis of neural networks. We validate our theoretical results in various numerical experiments on classification and regression tasks using real-world data.