Where Does the Robustness Come from? A Study of the Transformation-based Ensemble Defence

TL;DR

This study investigates the true source of robustness in transformation-based ensemble defenses for image classification, revealing that irreversible transformations, not ensemble size, primarily contribute to robustness.

Contribution

It provides a detailed analysis showing that robustness mainly stems from irreversible transformations rather than ensemble effects or increasing sub-models.

Findings

Transferability exists among models trained on transformed data

Robustness from transformations is limited

Irreversible transformations are the main robustness source

Abstract

This paper aims to provide a thorough study on the effectiveness of the transformation-based ensemble defence for image classification and its reasons. It has been empirically shown that they can enhance the robustness against evasion attacks, while there is little analysis on the reasons. In particular, it is not clear whether the robustness improvement is a result of transformation or ensemble. In this paper, we design two adaptive attacks to better evaluate the transformation-based ensemble defence. We conduct experiments to show that 1) the transferability of adversarial examples exists among the models trained on data records after different reversible transformations; 2) the robustness gained through transformation-based ensemble is limited; 3) this limited robustness is mainly from the irreversible transformations rather than the ensemble of a number of models; and 4) blindly increasing the number of sub-models in a transformation-based ensemble does not bring extra robustness gain.