A general framework for defining and optimizing robustness

TL;DR

This paper introduces a comprehensive, mathematically rigorous framework for defining and optimizing various robustness properties of classifiers, applicable across different models and robustness objectives.

Contribution

It proposes a general, model-agnostic robustness framework based on foundational postulates, and develops new neural network co-training methods for specific robustness goals.

Findings

Framework applicable to any classification model

New co-training strategies for robustness optimization

Addresses robustness from safety to transferability

Abstract

Robustness of neural networks has recently attracted a great amount of interest. The many investigations in this area lack a precise common foundation of robustness concepts. Therefore, in this paper, we propose a rigorous and flexible framework for defining different types of robustness properties for classifiers. Our robustness concept is based on postulates that robustness of a classifier should be considered as a property that is independent of accuracy, and that it should be defined in purely mathematical terms without reliance on algorithmic procedures for its measurement. We develop a very general robustness framework that is applicable to any type of classification model, and that encompasses relevant robustness concepts for investigations ranging from safety against adversarial attacks to transferability of models to new domains. For two prototypical, distinct robustness objectives we then propose new learning approaches based on neural network co-training strategies for obtaining image classifiers optimized for these respective objectives.