Adversarial Imaging Pipelines

TL;DR

This paper introduces a novel adversarial attack method targeting specific camera image processing pipelines, revealing how ISP transformations can be exploited to deceive neural network classifiers with high success rates.

Contribution

It develops a differentiable multi-task optimization approach to craft adversarial patterns that target specific camera ISPs, accounting for the entire image processing pipeline.

Findings

Achieves 92% fooling rate on automotive hardware ISPs.

Demonstrates 90% fooling rate with physical optics attacks.

Reveals vulnerability of camera-specific image processing pipelines.

Abstract

Adversarial attacks play an essential role in understanding deep neural network predictions and improving their robustness. Existing attack methods aim to deceive convolutional neural network (CNN)-based classifiers by manipulating RGB images that are fed directly to the classifiers. However, these approaches typically neglect the influence of the camera optics and image processing pipeline (ISP) that produce the network inputs. ISPs transform RAW measurements to RGB images and traditionally are assumed to preserve adversarial patterns. However, these low-level pipelines can, in fact, destroy, introduce or amplify adversarial patterns that can deceive a downstream detector. As a result, optimized patterns can become adversarial for the classifier after being transformed by a certain camera ISP and optic but not for others. In this work, we examine and develop such an attack that deceives a specific camera ISP while leaving others intact, using the same down-stream classifier. We frame camera-specific attacks as a multi-task optimization problem, relying on a differentiable approximation for the ISP itself. We validate the proposed method using recent state-of-the-art automotive hardware ISPs, achieving 92% fooling rate when attacking a specific ISP. We demonstrate physical optics attacks with 90% fooling rate for a specific camera lenses.