CertViT: Certified Robustness of Pre-Trained Vision Transformers

TL;DR

CertViT introduces a scalable method to achieve certified robustness in large pre-trained vision transformers by fine-tuning with a two-step proximal-projection approach, improving robustness without extensive retraining.

Contribution

The paper proposes CertViT, a novel two-step proximal-projection method enabling certified robustness of large pre-trained vision transformers without full retraining.

Findings

CertViT outperforms state-of-the-art Lipschitz-trained networks in certified accuracy.

The method effectively enhances adversarial robustness of various vision transformer variants.

CertViT maintains high clean accuracy while providing robustness guarantees.

Abstract

Lipschitz bounded neural networks are certifiably robust and have a good trade-off between clean and certified accuracy. Existing Lipschitz bounding methods train from scratch and are limited to moderately sized networks (< 6M parameters). They require a fair amount of hyper-parameter tuning and are computationally prohibitive for large networks like Vision Transformers (5M to 660M parameters). Obtaining certified robustness of transformers is not feasible due to the non-scalability and inflexibility of the current methods. This work presents CertViT, a two-step proximal-projection method to achieve certified robustness from pre-trained weights. The proximal step tries to lower the Lipschitz bound and the projection step tries to maintain the clean accuracy of pre-trained weights. We show that CertViT networks have better certified accuracy than state-of-the-art Lipschitz trained networks. We apply CertViT on several variants of pre-trained vision transformers and show adversarial robustness using standard attacks. Code : https://github.com/sagarverma/transformer-lipschitz