Sparse Black-box Video Attack with Reinforcement Learning

TL;DR

This paper introduces a reinforcement learning-based method for black-box video attacks that adaptively selects key frames based on attack feedback, resulting in smaller perturbations and fewer queries.

Contribution

It formulates black-box video attacks as a reinforcement learning problem, enabling dynamic frame selection that improves attack efficiency and effectiveness.

Findings

Significantly reduces adversarial perturbations.

Achieves efficient query times on benchmark datasets.

Effective against C3D and LRCN models.

Abstract

Adversarial attacks on video recognition models have been explored recently. However, most existing works treat each video frame equally and ignore their temporal interactions. To overcome this drawback, a few methods try to select some key frames and then perform attacks based on them. Unfortunately, their selection strategy is independent of the attacking step, therefore the resulting performance is limited. Instead, we argue the frame selection phase is closely relevant with the attacking phase. The key frames should be adjusted according to the attacking results. For that, we formulate the black-box video attacks into a Reinforcement Learning (RL) framework. Specifically, the environment in RL is set as the recognition model, and the agent in RL plays the role of frame selecting. By continuously querying the recognition models and receiving the attacking feedback, the agent gradually adjusts its frame selection strategy and adversarial perturbations become smaller and smaller. We conduct a series of experiments with two mainstream video recognition models: C3D and LRCN on the public UCF-101 and HMDB-51 datasets. The results demonstrate that the proposed method can significantly reduce the adversarial perturbations with efficient query times.