Learning Image Attacks toward Vision Guided Autonomous Vehicles

TL;DR

This paper introduces a real-time online adversarial attack framework for autonomous vehicles that leverages reinforcement learning and a state estimator to effectively mislead vehicle missions in dynamic environments.

Contribution

It develops a reinforcement learning-based generative model for real-time image attacks that do not require full optimization at each frame, considering physical dynamics.

Findings

Effective misguidance of autonomous vehicle missions demonstrated in simulation.

The attack policy is robust to environmental uncertainties and physical variables.

Real-time attack generation without full optimization at each frame.

Abstract

While adversarial neural networks have been shown successful for static image attacks, very few approaches have been developed for attacking online image streams while taking into account the underlying physical dynamics of autonomous vehicles, their mission, and environment. This paper presents an online adversarial machine learning framework that can effectively misguide autonomous vehicles' missions. In the existing image attack methods devised toward autonomous vehicles, optimization steps are repeated for every image frame. This framework removes the need for fully converged optimization at every frame to realize image attacks in real-time. Using reinforcement learning, a generative neural network is trained over a set of image frames to obtain an attack policy that is more robust to dynamic and uncertain environments. A state estimator is introduced for processing image streams to reduce the attack policy's sensitivity to physical variables such as unknown position and velocity. A simulation study is provided to validate the results.