Adversarial Contrastive Learning via Asymmetric InfoNCE

TL;DR

This paper introduces A-InfoNCE, an asymmetric contrastive learning method that improves adversarial robustness by treating adversarial samples differently, effectively mitigating conflicts between contrastive learning and adversarial training.

Contribution

It proposes an asymmetric InfoNCE objective that discriminates adversarial samples, enhancing adversarial robustness without extra computational cost.

Findings

Outperforms existing adversarial contrastive learning methods

Consistent improvements across different finetuning schemes

A-InfoNCE can be extended to other contrastive learning approaches

Abstract

Contrastive learning (CL) has recently been applied to adversarial learning tasks. Such practice considers adversarial samples as additional positive views of an instance, and by maximizing their agreements with each other, yields better adversarial robustness. However, this mechanism can be potentially flawed, since adversarial perturbations may cause instance-level identity confusion, which can impede CL performance by pulling together different instances with separate identities. To address this issue, we propose to treat adversarial samples unequally when contrasted, with an asymmetric InfoNCE objective ($A-InfoNCE$) that allows discriminating considerations of adversarial samples. Specifically, adversaries are viewed as inferior positives that induce weaker learning signals, or as hard negatives exhibiting higher contrast to other negative samples. In the asymmetric fashion, the adverse impacts of conflicting objectives between CL and adversarial learning can be effectively mitigated. Experiments show that our approach consistently outperforms existing Adversarial CL methods across different finetuning schemes without additional computational cost. The proposed A-InfoNCE is also a generic form that can be readily extended to other CL methods. Code is available at https://github.com/yqy2001/A-InfoNCE.