Attacks and Faults Injection in Self-Driving Agents on the Carla Simulator -- Experience Report

TL;DR

This paper reports on experiments injecting adversarial attacks and faults into a self-driving agent in a simulator, demonstrating safety risks and emphasizing the need for protective measures and extensive testing.

Contribution

It provides a reproducible methodology for testing self-driving agents against attacks and faults using open-source tools and simulators.

Findings

Injected attacks cause erroneous decisions in the agent.

Faults can severely jeopardize vehicle safety.

The approach is feasible and easily reproducible.

Abstract

Machine Learning applications are acknowledged at the foundation of autonomous driving, because they are the enabling technology for most driving tasks. However, the inclusion of trained agents in automotive systems exposes the vehicle to novel attacks and faults, that can result in safety threats to the driv-ing tasks. In this paper we report our experimental campaign on the injection of adversarial attacks and software faults in a self-driving agent running in a driving simulator. We show that adversarial attacks and faults injected in the trained agent can lead to erroneous decisions and severely jeopardize safety. The paper shows a feasible and easily-reproducible approach based on open source simula-tor and tools, and the results clearly motivate the need of both protective measures and extensive testing campaigns.