Lipschitz constant estimation for 1D convolutional neural networks

TL;DR

This paper introduces a novel dissipativity-based approach for estimating Lipschitz constants of 1D CNNs, leveraging state space representations and semidefinite programming to improve accuracy and scalability.

Contribution

It presents a new method combining dissipativity theory and state space analysis for efficient Lipschitz constant estimation of 1D CNNs.

Findings

Lipschitz bounds are more accurate than existing methods.

The approach scales well to larger networks.

The method outperforms previous techniques in efficiency.

Abstract

In this work, we propose a dissipativity-based method for Lipschitz constant estimation of 1D convolutional neural networks (CNNs). In particular, we analyze the dissipativity properties of convolutional, pooling, and fully connected layers making use of incremental quadratic constraints for nonlinear activation functions and pooling operations. The Lipschitz constant of the concatenation of these mappings is then estimated by solving a semidefinite program which we derive from dissipativity theory. To make our method as efficient as possible, we exploit the structure of convolutional layers by realizing these finite impulse response filters as causal dynamical systems in state space and carrying out the dissipativity analysis for the state space realizations. The examples we provide show that our Lipschitz bounds are advantageous in terms of accuracy and scalability.