TACO: Adversarial Camouflage Optimization on Trucks to Fool Object Detectors

TL;DR

TACO is a novel framework that creates adversarial camouflage patterns on trucks using differentiable rendering to fool advanced object detectors like YOLOv8, significantly reducing detection accuracy and transferring across models.

Contribution

The paper introduces TACO, a new method combining Unreal Engine 5 and differentiable rendering to generate effective, visually plausible adversarial textures for trucks.

Findings

TACO reduces YOLOv8 detection AP@0.5 to 0.0099 on test data.

Adversarial patterns transfer effectively to Faster R-CNN and earlier YOLO versions.

The approach demonstrates significant degradation in object detection performance.

Abstract

Adversarial attacks threaten the reliability of machine learning models in critical applications like autonomous vehicles and defense systems. As object detectors become more robust with models like YOLOv8, developing effective adversarial methodologies is increasingly challenging. We present Truck Adversarial Camouflage Optimization (TACO), a novel framework that generates adversarial camouflage patterns on 3D vehicle models to deceive state-of-the-art object detectors. Adopting Unreal Engine 5, TACO integrates differentiable rendering with a Photorealistic Rendering Network to optimize adversarial textures targeted at YOLOv8. To ensure the generated textures are both effective in deceiving detectors and visually plausible, we introduce the Convolutional Smooth Loss function, a generalized smooth loss function. Experimental evaluations demonstrate that TACO significantly degrades YOLOv8's detection performance, achieving an AP@0.5 of 0.0099 on unseen test data. Furthermore, these adversarial patterns exhibit strong transferability to other object detection models such as Faster R-CNN and earlier YOLO versions.