How Real Is Real? A Human Evaluation Framework for Unrestricted Adversarial Examples

TL;DR

This paper introduces SCOOTER, a human evaluation framework designed to assess the perceptual realism of unrestricted adversarial examples in images, addressing a gap in existing evaluation methods.

Contribution

The paper presents SCOOTER, a standardized, statistically rigorous human assessment framework for evaluating the perceptual quality of unrestricted adversarial images.

Findings

SCOOTER enables consistent human evaluation of adversarial image realism.

It provides guidelines and tools for conducting statistically significant experiments.

The framework helps determine if unrestricted attacks are truly imperceptible.

Abstract

With an ever-increasing reliance on machine learning (ML) models in the real world, adversarial examples threaten the safety of AI-based systems such as autonomous vehicles. In the image domain, they represent maliciously perturbed data points that look benign to humans (i.e., the image modification is not noticeable) but greatly mislead state-of-the-art ML models. Previously, researchers ensured the imperceptibility of their altered data points by restricting perturbations via $\ell_p$ norms. However, recent publications claim that creating natural-looking adversarial examples without such restrictions is also possible. With much more freedom to instill malicious information into data, these unrestricted adversarial examples can potentially overcome traditional defense strategies as they are not constrained by the limitations or patterns these defenses typically recognize and mitigate. This allows attackers to operate outside of expected threat models. However, surveying existing image-based methods, we noticed a need for more human evaluations of the proposed image modifications. Based on existing human-assessment frameworks for image generation quality, we propose SCOOTER - an evaluation framework for unrestricted image-based attacks. It provides researchers with guidelines for conducting statistically significant human experiments, standardized questions, and a ready-to-use implementation. We propose a framework that allows researchers to analyze how imperceptible their unrestricted attacks truly are.