# Learning from Peers: Collaborative Ensemble Adversarial Training

## TL;DR

This paper introduces CEAT, a collaborative ensemble adversarial training method that improves model robustness by emphasizing samples with high predictive disparity among sub-models, leading to state-of-the-art results.

## Contribution

The paper proposes a novel cooperative training approach that adaptively weights samples based on predictive disparities, enhancing ensemble robustness against adversarial attacks.

## Key findings

- CEAT achieves state-of-the-art robustness on benchmark datasets.
- It is model-agnostic and adaptable to various ensemble methods.
- CEAT outperforms existing EAT strategies in robustness metrics.

## Abstract

Ensemble Adversarial Training (EAT) attempts to enhance the robustness of models against adversarial attacks by leveraging multiple models. However, current EAT strategies tend to train the sub-models independently, ignoring the cooperative benefits between sub-models. Through detailed inspections of the process of EAT, we find that that samples with classification disparities between sub-models are close to the decision boundary of ensemble, exerting greater influence on the robustness of ensemble. To this end, we propose a novel yet efficient Collaborative Ensemble Adversarial Training (CEAT), to highlight the cooperative learning among sub-models in the ensemble. To be specific, samples with larger predictive disparities between the sub-models will receive greater attention during the adversarial training of the other sub-models. CEAT leverages the probability disparities to adaptively assign weights to different samples, by incorporating a calibrating distance regularization. Extensive experiments on widely-adopted datasets show that our proposed method achieves the state-of-the-art performance over competitive EAT methods. It is noteworthy that CEAT is model-agnostic, which can be seamlessly adapted into various ensemble methods with flexible applicability.