GenoArmory: A Unified Evaluation Framework for Adversarial Attacks on Genomic Foundation Models

TL;DR

GenoArmory introduces the first comprehensive benchmark for evaluating the vulnerability of genomic foundation models to adversarial attacks, providing insights into model robustness and biological significance of attack targets.

Contribution

It presents a unified evaluation framework and a new adversarial dataset for assessing and improving the robustness of GFMs against adversarial threats.

Findings

Classification models are more robust than generative models.

Adversarial attacks often target biologically important genomic regions.

The framework analyzes vulnerabilities across architectures, quantization, and datasets.

Abstract

We propose the first unified adversarial attack benchmark for Genomic Foundation Models (GFMs), named GenoArmory. Unlike existing GFM benchmarks, GenoArmory offers the first comprehensive evaluation framework to systematically assess the vulnerability of GFMs to adversarial attacks. Methodologically, we evaluate the adversarial robustness of five state-of-the-art GFMs using four widely adopted attack algorithms and three defense strategies. Importantly, our benchmark provides an accessible and comprehensive framework to analyze GFM vulnerabilities with respect to model architecture, quantization schemes, and training datasets. Additionally, we introduce GenoAdv, a new adversarial sample dataset designed to improve GFM safety. Empirically, classification models exhibit greater robustness to adversarial perturbations compared to generative models, highlighting the impact of task type on model vulnerability. Moreover, adversarial attacks frequently target biologically significant genomic regions, suggesting that these models effectively capture meaningful sequence features.