# Adversarially robust transfer learning

## TL;DR

This paper explores transfer learning methods to develop neural networks that are both accurate and adversarially robust, emphasizing strategies that preserve robustness during adaptation with limited data and computational resources.

## Contribution

It introduces robust transfer learning techniques that inherit robustness from source models and proposes lifelong learning strategies to maintain robustness during fine-tuning.

## Key findings

- Robust feature extractors enable transfer of robustness.
- Lifelong learning preserves robustness during fine-tuning.
- Strategies improve generalization of adversarially trained models.

## Abstract

Transfer learning, in which a network is trained on one task and re-purposed on another, is often used to produce neural network classifiers when data is scarce or full-scale training is too costly. When the goal is to produce a model that is not only accurate but also adversarially robust, data scarcity and computational limitations become even more cumbersome. We consider robust transfer learning, in which we transfer not only performance but also robustness from a source model to a target domain. We start by observing that robust networks contain robust feature extractors. By training classifiers on top of these feature extractors, we produce new models that inherit the robustness of their parent networks. We then consider the case of fine tuning a network by re-training end-to-end in the target domain. When using lifelong learning strategies, this process preserves the robustness of the source network while achieving high accuracy. By using such strategies, it is possible to produce accurate and robust models with little data, and without the cost of adversarial training. Additionally, we can improve the generalization of adversarially trained models, while maintaining their robustness.