Adversarial Contrastive Learning by Permuting Cluster Assignments

TL;DR

SwARo introduces a novel adversarial contrastive learning framework that uses cluster assignment permutations to enhance robustness, semantic similarity, and efficiency in self-supervised learning, showing consistent improvements over existing methods.

Contribution

This work is the first to jointly address robustness, semantic clustering, and computational efficiency through adversarial cluster assignment permutations in contrastive learning.

Findings

SwARo outperforms state-of-the-art methods on multiple benchmarks.

It demonstrates improved robustness against various attacks.

The approach enhances semantic clustering and reduces computational costs.

Abstract

Contrastive learning has gained popularity as an effective self-supervised representation learning technique. Several research directions improve traditional contrastive approaches, e.g., prototypical contrastive methods better capture the semantic similarity among instances and reduce the computational burden by considering cluster prototypes or cluster assignments, while adversarial instance-wise contrastive methods improve robustness against a variety of attacks. To the best of our knowledge, no prior work jointly considers robustness, cluster-wise semantic similarity and computational efficiency. In this work, we propose SwARo, an adversarial contrastive framework that incorporates cluster assignment permutations to generate representative adversarial samples. We evaluate SwARo on multiple benchmark datasets and against various white-box and black-box attacks, obtaining consistent improvements over state-of-the-art baselines.