Regularized Ensembles and Transferability in Adversarial Learning

TL;DR

This paper investigates how regularization techniques and ensemble methods affect the transferability and robustness of neural networks against adversarial attacks, revealing that certain regularizers and partial ensemble information hinder transferability.

Contribution

It introduces an analysis of the impact of regularization and ensemble composition on adversarial transferability, highlighting barriers to transferability in regularized models and ensembles.

Findings

Regularization can create barriers to transferability.

Partial ensemble information reduces transferability.

Different regularizers impact model robustness against adversarial examples.

Abstract

Despite the considerable success of convolutional neural networks in a broad array of domains, recent research has shown these to be vulnerable to small adversarial perturbations, commonly known as adversarial examples. Moreover, such examples have shown to be remarkably portable, or transferable, from one model to another, enabling highly successful black-box attacks. We explore this issue of transferability and robustness from two dimensions: first, considering the impact of conventional $l_p$ regularization as well as replacing the top layer with a linear support vector machine (SVM), and second, the value of combining regularized models into an ensemble. We show that models trained with different regularizers present barriers to transferability, as does partial information about the models comprising the ensemble.