Online Evasion Attacks on Recurrent Models:The Power of Hallucinating the Future

TL;DR

This paper introduces a versatile attack framework for recurrent models in online tasks, highlighting their vulnerability by proposing a novel 'hallucinating' future attack that approximates clairvoyant performance.

Contribution

It presents a general online attack framework and a new Predictive Attack method that effectively 'hallucinates' future inputs, advancing understanding of model robustness in online settings.

Findings

Predictive Attack achieves 98% of clairvoyant attack performance

Framework covers time-varying objectives and constraints

Validated through extensive experiments

Abstract

Recurrent models are frequently being used in online tasks such as autonomous driving, and a comprehensive study of their vulnerability is called for. Existing research is limited in generality only addressing application-specific vulnerability or making implausible assumptions such as the knowledge of future input. In this paper, we present a general attack framework for online tasks incorporating the unique constraints of the online setting different from offline tasks. Our framework is versatile in that it covers time-varying adversarial objectives and various optimization constraints, allowing for a comprehensive study of robustness. Using the framework, we also present a novel white-box attack called Predictive Attack that `hallucinates' the future. The attack achieves 98 percent of the performance of the ideal but infeasible clairvoyant attack on average. We validate the effectiveness of the proposed framework and attacks through various experiments.