An Empirical Study of Accuracy-Robustness Tradeoff and Training Efficiency in Self-Supervised Learning

TL;DR

This paper introduces a crop-based robust EMP-SSL framework that accelerates training and improves the balance between accuracy and robustness, and extends it with free adversarial training to further enhance efficiency and performance.

Contribution

It presents a novel multi-crop sampling approach in SSL that reduces training epochs and introduces CF-AMC-SSL with free adversarial training for improved efficiency and robustness.

Findings

Accelerates SSL convergence with crop-based sampling

Achieves better accuracy-robustness tradeoff than previous methods

Reduces training time using free adversarial training

Abstract

Self-supervised learning (SSL) has significantly advanced image representation learning, yet efficiency challenges persist, particularly with adversarial training. Many SSL methods require extensive epochs to achieve convergence, a demand further amplified in adversarial settings. To address this inefficiency, we revisit the robust EMP-SSL framework, emphasizing the importance of increasing the number of crops per image to accelerate learning. Unlike traditional contrastive learning, robust EMP-SSL leverages multi-crop sampling, integrates an invariance term and regularization, and reduces training epochs, enhancing time efficiency. Evaluated with both standard linear classifiers and multi-patch embedding aggregation, robust EMP-SSL provides new insights into SSL evaluation strategies. Our results show that robust crop-based EMP-SSL not only accelerates convergence but also achieves a superior balance between clean accuracy and adversarial robustness, outperforming multi-crop embedding aggregation. Additionally, we extend this approach with free adversarial training in Multi-Crop SSL, introducing the Cost-Free Adversarial Multi-Crop Self-Supervised Learning (CF-AMC-SSL) method. CF-AMC-SSL demonstrates the effectiveness of free adversarial training in reducing training time while simultaneously improving clean accuracy and adversarial robustness. These findings underscore the potential of CF-AMC-SSL for practical SSL applications. Our code is publicly available at https://github.com/softsys4ai/CF-AMC-SSL.