Ortho-ODE: Enhancing Robustness and of Neural ODEs against Adversarial Attacks

TL;DR

This paper investigates the inherent robustness of Neural ODEs against adversarial attacks, showing that controlling the Lipschitz constant enhances their resilience, supported by theoretical derivations and experiments on multiple datasets.

Contribution

It introduces a method to improve Neural ODE robustness by regulating the Lipschitz constant using Grownwall's inequality, linking contractivity theory to adversarial robustness.

Findings

Lipschitz control significantly boosts robustness.

The approach improves performance on MNIST, CIFAR-10, and CIFAR-100.

Adaptive solvers influence the robustness of Neural ODEs.

Abstract

Neural Ordinary Differential Equations (NODEs) probed the usage of numerical solvers to solve the differential equation characterized by a Neural Network (NN), therefore initiating a new paradigm of deep learning models with infinite depth. NODEs were designed to tackle the irregular time series problem. However, NODEs have demonstrated robustness against various noises and adversarial attacks. This paper is about the natural robustness of NODEs and examines the cause behind such surprising behaviour. We show that by controlling the Lipschitz constant of the ODE dynamics the robustness can be significantly improved. We derive our approach from Grownwall's inequality. Further, we draw parallels between contractivity theory and Grownwall's inequality. Experimentally we corroborate the enhanced robustness on numerous datasets - MNIST, CIFAR-10, and CIFAR 100. We also present the impact of adaptive and non-adaptive solvers on the robustness of NODEs.