Robust Machine Learning Modeling for Predictive Control Using Lipschitz-Constrained Neural Networks

TL;DR

This paper introduces Lipschitz-Constrained Neural Networks (LCNNs) with spectral layers for robust nonlinear system modeling in predictive control, demonstrating improved robustness and generalization over traditional neural networks.

Contribution

The paper develops LCNNs with spectral layers, proves their universal approximation and generalization bounds, and integrates them into MPC for enhanced robustness against data noise.

Findings

LCNNs can approximate 1-Lipschitz functions effectively.

LCNN-based MPC outperforms conventional NNs in noisy data scenarios.

Theoretical bounds support LCNNs' robustness and generalization.

Abstract

Neural networks (NNs) have emerged as a state-of-the-art method for modeling nonlinear systems in model predictive control (MPC). However, the robustness of NNs, in terms of sensitivity to small input perturbations, remains a critical challenge for practical applications. To address this, we develop Lipschitz-Constrained Neural Networks (LCNNs) for modeling nonlinear systems and derive rigorous theoretical results to demonstrate their effectiveness in approximating Lipschitz functions, reducing input sensitivity, and preventing over-fitting. Specifically, we first prove a universal approximation theorem to show that LCNNs using SpectralDense layers can approximate any 1-Lipschitz target function. Then, we prove a probabilistic generalization error bound for LCNNs using SpectralDense layers by using their empirical Rademacher complexity. Finally, the LCNNs are incorporated into the MPC scheme, and a chemical process example is utilized to show that LCNN-based MPC outperforms MPC using conventional feedforward NNs in the presence of training data noise.