TsallisPGD: Adaptive Gradient Weighting for Adversarial Attacks on Semantic Segmentation

TL;DR

TsallisPGD introduces a novel adversarial attack for semantic segmentation that adaptively adjusts gradient emphasis across pixels using a dynamic Tsallis cross-entropy, outperforming existing methods.

Contribution

The paper proposes TsallisPGD, a new adaptive attack leveraging a dynamic Tsallis cross-entropy to improve attack effectiveness on semantic segmentation models.

Findings

TsallisPGD achieves the best average attack rank across multiple datasets and models.

It outperforms existing attack methods like CEPGD, SegPGD, CosPGD, JSPGD, and MaskedPGD.

Using a dynamic q-schedule enhances attack performance over fixed q-values.

Abstract

Attacking semantic segmentation models is significantly harder than image classification models because an attacker must flip thousands of pixel predictions simultaneously. Standard pixel-wise cross-entropy (CE) is ill-suited to this setting: it tends to overemphasize already-misclassified pixels, which slows optimization and overstates model robustness. To address these issues, we introduce TsallisPGD, an adversarial attack built on the Tsallis cross-entropy, a generalization of CE parameterized by $q$, which adaptively reshapes the gradient landscape by controlling gradient concentration across pixels. By varying $q$, we steer the attack toward pixels at different confidence levels. We first show that no single fixed-$q$ is universally optimal, as its effectiveness depends on the dataset, model architecture, and perturbation budget. Motivated by this, we propose a dynamic $q$-schedule that sweeps $q$ during optimization. Extensive experiments on Cityscapes, Pascal VOC, and ADE20K show that TsallisPGD, using a single validation-selected schedule, achieves the best average attack rank across all evaluated settings and improves over CEPGD, SegPGD, CosPGD, JSPGD, and MaskedPGD in reducing accuracy and mIoU on both standard and robust models.