PivotAttack: Rethinking the Search Trajectory in Hard-Label Text Attacks via Pivot Words

TL;DR

PivotAttack introduces an efficient inside-out method using multi-armed bandits to identify key token groups, significantly improving hard-label text attack success rates and reducing query costs.

Contribution

It presents a novel query-efficient framework that leverages pivot sets and multi-armed bandits for more effective hard-label text attacks.

Findings

Outperforms state-of-the-art baselines in success rate

Reduces query costs significantly

Effective on both traditional and large language models

Abstract

Existing hard-label text attacks often rely on inefficient "outside-in" strategies that traverse vast search spaces. We propose PivotAttack, a query-efficient "inside-out" framework. It employs a Multi-Armed Bandit algorithm to identify Pivot Sets-combinatorial token groups acting as prediction anchors-and strategically perturbs them to induce label flips. This approach captures inter-word dependencies and minimizes query costs. Extensive experiments across traditional models and Large Language Models demonstrate that PivotAttack consistently outperforms state-of-the-art baselines in both Attack Success Rate and query efficiency.