Improving the robustness of neural ODEs with minimal weight perturbation

TL;DR

This paper introduces a method to improve neural ODE stability by minimal weight perturbations, enhancing robustness against initial value perturbations and adversarial attacks through an eigenvalue optimization-based training strategy.

Contribution

It presents a novel eigenvalue optimization algorithm to control the Jacobian's logarithmic norm, thereby increasing neural ODE stability and robustness during training.

Findings

Neural ODE-based classifiers become more stable with the proposed method.

The approach reduces vulnerability to adversarial attacks.

Numerical experiments confirm improved robustness on image datasets.

Abstract

We propose a method to enhance the stability of a neural ordinary differential equation (neural ODE) by reducing the maximum error growth subsequent to a perturbation of the initial value. Since the stability depends on the logarithmic norm of the Jacobian matrix associated with the neural ODE, we control the logarithmic norm by perturbing the weight matrices of the neural ODE by a smallest possible perturbation (in Frobenius norm). We do so by engaging an eigenvalue optimisation problem, for which we propose a nested two-level algorithm. For a given perturbation size of the weight matrix, the inner level computes optimal perturbations of that size, while - at the outer level - we tune the perturbation amplitude until we reach the desired uniform stability bound. We embed the proposed algorithm in the training of the neural ODE to improve its robustness to perturbations of the initial value, as adversarial attacks. Numerical experiments on classical image datasets show that an image classifier including a neural ODE in its architecture trained according to our strategy is more stable than the same classifier trained in the classical way, and therefore, it is more robust and less vulnerable to adversarial attacks.