Exposing and Defending the Achilles' Heel of Video Mixture-of-Experts

TL;DR

This paper investigates the adversarial vulnerabilities of video Mixture-of-Experts models at the component level, proposing targeted attacks and defenses that improve robustness and reduce inference costs.

Contribution

It introduces Temporal Lipschitz-Guided Attacks and joint adversarial training to expose and defend against component-wise and collaborative weaknesses in video MoE models.

Findings

Joint attacks significantly increase adversarial effects.

Proposed defenses improve robustness across datasets.

Inference cost is reduced by over 60%.

Abstract

Mixture-of-Experts (MoE) has demonstrated strong performance in video understanding tasks, yet its adversarial robustness remains underexplored. Existing attack methods often treat MoE as a unified architecture, overlooking the independent and collaborative weaknesses of key components such as routers and expert modules. To fill this gap, we propose Temporal Lipschitz-Guided Attacks (TLGA) to thoroughly investigate component-level vulnerabilities in video MoE models. We first design attacks on the router, revealing its independent weaknesses. Building on this, we introduce Joint Temporal Lipschitz-Guided Attacks (J-TLGA), which collaboratively perturb both routers and experts. This joint attack significantly amplifies adversarial effects and exposes the Achilles' Heel (collaborative weaknesses) of the MoE architecture. Based on these insights, we further propose Joint Temporal Lipschitz Adversarial Training (J-TLAT). J-TLAT performs joint training to further defend against collaborative weaknesses, enhancing component-wise robustness. Our framework is plug-and-play and reduces inference cost by more than 60% compared with dense models. It consistently enhances adversarial robustness across diverse datasets and architectures, effectively mitigating both the independent and collaborative weaknesses of MoE.