HO-FMN: Hyperparameter Optimization for Fast Minimum-Norm Attacks

TL;DR

This paper introduces HO-FMN, a flexible hyperparameter optimization framework for fast minimum-norm attacks, enabling more accurate robustness evaluation of machine learning models by dynamically adjusting attack parameters.

Contribution

It proposes a parametric variation of the attack algorithm allowing dynamic adjustment of loss, optimizer, and hyperparameters, improving attack effectiveness without extra tuning.

Findings

The attack finds smaller adversarial perturbations compared to fixed-parameter methods.

It enables robustness evaluation as a function of perturbation budget.

The method remains efficient despite increased flexibility.

Abstract

Gradient-based attacks are a primary tool to evaluate robustness of machine-learning models. However, many attacks tend to provide overly-optimistic evaluations as they use fixed loss functions, optimizers, step-size schedulers, and default hyperparameters. In this work, we tackle these limitations by proposing a parametric variation of the well-known fast minimum-norm attack algorithm, whose loss, optimizer, step-size scheduler, and hyperparameters can be dynamically adjusted. We re-evaluate 12 robust models, showing that our attack finds smaller adversarial perturbations without requiring any additional tuning. This also enables reporting adversarial robustness as a function of the perturbation budget, providing a more complete evaluation than that offered by fixed-budget attacks, while remaining efficient. We release our open-source code at https://github.com/pralab/HO-FMN.