A Tale of Two Cities: Data and Configuration Variances in Robust Deep Learning

TL;DR

This paper examines how data and configuration variances affect the robustness of deep neural networks, proposing a holistic framework to generate representative variances for improving model robustness.

Contribution

It introduces a comprehensive view of DNN robustness considering both data and configuration variances and presents a predictive framework for generating counterexamples.

Findings

Holistic view of data and configuration variances in DNN robustness

Framework for generating representative variances using search-based optimization

Emphasis on robustness as a priority for real-world deployment

Abstract

Deep neural networks (DNNs), are widely used in many industries such as image recognition, supply chain, medical diagnosis, and autonomous driving. However, prior work has shown the high accuracy of a DNN model does not imply high robustness (i.e., consistent performances on new and future datasets) because the input data and external environment (e.g., software and model configurations) for a deployed model are constantly changing. Hence, ensuring the robustness of deep learning is not an option but a priority to enhance business and consumer confidence. Previous studies mostly focus on the data aspect of model variance. In this article, we systematically summarize DNN robustness issues and formulate them in a holistic view through two important aspects, i.e., data and software configuration variances in DNNs. We also provide a predictive framework to generate representative variances (counterexamples) by considering both data and configurations for robust learning through the lens of search-based optimization.