NICGSlowDown: Evaluating the Efficiency Robustness of Neural Image Caption Generation Models

TL;DR

This paper introduces NICGSlowDown, an attack method that subtly alters images to significantly increase the computational latency of neural image caption generation models, highlighting a new efficiency vulnerability.

Contribution

The paper presents NICGSlowDown, a novel attack approach to evaluate and demonstrate the efficiency robustness issues in NICG models under minimal input perturbations.

Findings

NICGSlowDown can increase model latency by up to 483.86%.

Perturbations are human-unnoticeable.

Highlights the importance of efficiency robustness in NICG models.

Abstract

Neural image caption generation (NICG) models have received massive attention from the research community due to their excellent performance in visual understanding. Existing work focuses on improving NICG model accuracy while efficiency is less explored. However, many real-world applications require real-time feedback, which highly relies on the efficiency of NICG models. Recent research observed that the efficiency of NICG models could vary for different inputs. This observation brings in a new attack surface of NICG models, i.e., An adversary might be able to slightly change inputs to cause the NICG models to consume more computational resources. To further understand such efficiency-oriented threats, we propose a new attack approach, NICGSlowDown, to evaluate the efficiency robustness of NICG models. Our experimental results show that NICGSlowDown can generate images with human-unnoticeable perturbations that will increase the NICG model latency up to 483.86%. We hope this research could raise the community's concern about the efficiency robustness of NICG models.