Adversarial Patch Attacks on Vision-Based Cargo Occupancy Estimation via Differentiable 3D Simulation

TL;DR

This paper investigates the vulnerability of vision-based cargo occupancy estimation systems to physical adversarial patches using differentiable 3D simulation, demonstrating high success rates especially in denial-of-service attacks.

Contribution

It introduces the first study of adversarial patch attacks on cargo-occupancy estimation in realistic 3D simulations, optimizing patches across various scene variations.

Findings

3D-optimized patches achieve up to 84.94% success in denial-of-service attacks.

Concealment attacks reach a success rate of 30.32%.

Differentiable rendering enables effective adversarial patch optimization.

Abstract

Computer vision systems are increasingly adopted in modern logistics operations, including the estimation of trailer occupancy for planning, routing, and billing. Although effective, such systems may be vulnerable to physical adversarial attacks, particularly adversarial patches that can be printed and placed on interior surfaces. In this work, we study the feasibility of such attacks on a convolutional cargo-occupancy classifier using fully simulated 3D environments. Using Mitsuba 3 for differentiable rendering, we optimize patch textures across variations in geometry, lighting, and viewpoint, and compare their effectiveness to a 2D compositing baseline. Our experiments demonstrate that 3D-optimized patches achieve high attack success rates, especially in a denial-of-service scenario (empty to full), where success reaches 84.94 percent. Concealment attacks (full to empty) prove more challenging but still reach 30.32 percent. We analyze the factors influencing attack success, discuss implications for the security of automated logistics pipelines, and highlight directions for strengthening physical robustness. To our knowledge, this is the first study to investigate adversarial patch attacks for cargo-occupancy estimation in physically realistic, fully simulated 3D scenes.