SkeleVision: Towards Adversarial Resiliency of Person Tracking with Multi-Task Learning

TL;DR

This paper explores how multi-task learning with human keypoint detection improves the adversarial robustness of person tracking systems, making them more secure against realistic attacks.

Contribution

It demonstrates that joint training with related tasks enhances the adversarial resilience of SiamRPN trackers in person tracking applications.

Findings

MTL increases robustness against physical adversarial attacks

Joint learning with keypoint detection improves tracker security

Empirical results on real-world datasets confirm effectiveness

Abstract

Person tracking using computer vision techniques has wide ranging applications such as autonomous driving, home security and sports analytics. However, the growing threat of adversarial attacks raises serious concerns regarding the security and reliability of such techniques. In this work, we study the impact of multi-task learning (MTL) on the adversarial robustness of the widely used SiamRPN tracker, in the context of person tracking. Specifically, we investigate the effect of jointly learning with semantically analogous tasks of person tracking and human keypoint detection. We conduct extensive experiments with more powerful adversarial attacks that can be physically realizable, demonstrating the practical value of our approach. Our empirical study with simulated as well as real-world datasets reveals that training with MTL consistently makes it harder to attack the SiamRPN tracker, compared to typically training only on the single task of person tracking.