AdversariaLLM: A Unified and Modular Toolbox for LLM Robustness Research

TL;DR

AdversariaLLM is a comprehensive, modular toolbox designed to improve reproducibility, correctness, and comparability in LLM robustness research by integrating multiple attack algorithms, datasets, and evaluation tools.

Contribution

It introduces a unified framework for LLM robustness testing that consolidates attack methods, datasets, and evaluation metrics, addressing fragmentation and reproducibility issues.

Findings

Implemented 12 adversarial attack algorithms

Integrated 7 benchmark datasets for various evaluations

Provides reproducibility features like resource tracking and deterministic results

Abstract

The rapid expansion of research on Large Language Model (LLM) safety and robustness has produced a fragmented and oftentimes buggy ecosystem of implementations, datasets, and evaluation methods. This fragmentation makes reproducibility and comparability across studies challenging, hindering meaningful progress. To address these issues, we introduce AdversariaLLM, a toolbox for conducting LLM jailbreak robustness research. Its design centers on reproducibility, correctness, and extensibility. The framework implements twelve adversarial attack algorithms, integrates seven benchmark datasets spanning harmfulness, over-refusal, and utility evaluation, and provides access to a wide range of open-weight LLMs via Hugging Face. The implementation includes advanced features for comparability and reproducibility such as compute-resource tracking, deterministic results, and distributional evaluation techniques. \name also integrates judging through the companion package JudgeZoo, which can also be used independently. Together, these components aim to establish a robust foundation for transparent, comparable, and reproducible research in LLM safety.