Adversarial Interaction Attack: Fooling AI to Misinterpret Human Intentions

TL;DR

This paper demonstrates that deep learning AI systems can be easily deceived by adversarial noise in interaction scenarios, risking safety in human-AI collaborations, and introduces a novel attack method applicable beyond skeleton data.

Contribution

The paper introduces a new adversarial attack on interaction models, revealing vulnerabilities in AI systems that interpret human actions, with broad applicability to sequential regression problems.

Findings

Deep learning models can be fooled by subtle adversarial noise.

The attack successfully manipulates predictions of human reactions.

Potential risks in deploying AI in safety-critical interaction contexts.

Abstract

Understanding the actions of both humans and artificial intelligence (AI) agents is important before modern AI systems can be fully integrated into our daily life. In this paper, we show that, despite their current huge success, deep learning based AI systems can be easily fooled by subtle adversarial noise to misinterpret the intention of an action in interaction scenarios. Based on a case study of skeleton-based human interactions, we propose a novel adversarial attack on interactions, and demonstrate how DNN-based interaction models can be tricked to predict the participants' reactions in unexpected ways. From a broader perspective, the scope of our proposed attack method is not confined to problems related to skeleton data but can also be extended to any type of problems involving sequential regressions. Our study highlights potential risks in the interaction loop with AI and humans, which need to be carefully addressed when deploying AI systems in safety-critical applications.