Wasserstein Distributionally Robust Shallow Convex Neural Networks

TL;DR

This paper introduces WaDiRo-SCNNs, a convex neural network training method based on Wasserstein distributional robustness, providing reliable predictions under data corruption with guarantees and applicability to critical sectors.

Contribution

The paper develops a novel convex training framework for shallow ReLU neural networks using Wasserstein distributionally robust optimization, enabling scalable, safe, and verifiable models.

Findings

Demonstrates improved out-of-sample performance on synthetic and real-world data.

Enforces physical constraints within the training process.

Provides stability verification through a mixed-integer convex program.

Abstract

In this work, we propose Wasserstein distributionally robust shallow convex neural networks (WaDiRo-SCNNs) to provide reliable nonlinear predictions when subject to adverse and corrupted datasets. Our approach is based on the reformulation of a new convex training program for ReLU-based shallow neural networks, which allows us to cast the problem into the order-1 Wasserstein distributionally robust optimization framework. Our training procedure is conservative, has low stochasticity, is solvable with open-source solvers, and is scalable to large industrial deployments. We provide out-of-sample performance guarantees, show that hard convex physical constraints can be enforced in the training program, and propose a mixed-integer convex post-training verification program to evaluate model stability. WaDiRo-SCNN aims to make neural networks safer for critical applications, such as in the energy sector. Finally, we numerically demonstrate our model's performance through both a synthetic experiment and a real-world power system application, viz., the prediction of hourly energy consumption in non-residential buildings within the context of virtual power plants, and evaluate its stability across standard regression benchmark datasets. The experimental results are convincing and showcase the strengths of the proposed model.