Approach to Finding a Robust Deep Learning Model

TL;DR

This paper introduces a versatile approach and meta-algorithm for assessing and enhancing the robustness of deep learning models across various configurations and training conditions.

Contribution

It presents a novel, task-agnostic method for evaluating model robustness, including a model selection algorithm applicable to any suitable machine learning model.

Findings

Robustness varies with training sample size.

Initialization impacts model stability.

Inductive bias influences model resilience.

Abstract

The rapid development of machine learning (ML) and artificial intelligence (AI) applications requires the training of large numbers of models. This growing demand highlights the importance of training models without human supervision, while ensuring that their predictions are reliable. In response to this need, we propose a novel approach for determining model robustness. This approach, supplemented with a proposed model selection algorithm designed as a meta-algorithm, is versatile and applicable to any machine learning model, provided that it is appropriate for the task at hand. This study demonstrates the application of our approach to evaluate the robustness of deep learning models. To this end, we study small models composed of a few convolutional and fully connected layers, using common optimizers due to their ease of interpretation and computational efficiency. Within this framework, we address the influence of training sample size, model weight initialization, and inductive bias on the robustness of deep learning models.