Elytra: A Flexible Framework for Securing Large Vision Systems

TL;DR

Elytra introduces a flexible, efficient framework that uses low-rank adaptation to dynamically patch large vision models, significantly enhancing their robustness against adversarial attacks in autonomous systems.

Contribution

The paper presents Elytra, a novel framework employing low-rank adaptation for lightweight, dynamic security patches to improve large vision models' adversarial robustness.

Findings

Improves classification accuracy by up to 24.09% against adversarial examples.

Enables dynamic patching of large pre-trained vision models.

Offers a computationally efficient alternative to existing hardening methods.

Abstract

Adversarial attacks have emerged as a critical threat to autonomous driving systems. These attacks exploit the underlying neural network, allowing small, almost invisible, perturbations to alter the behavior of such systems in potentially malicious ways, e.g., causing a traffic sign classification network to misclassify a stop sign as a speed limit sign. Prior work in hardening such systems against adversarial attacks has looked at fine-tuning of the system or adding additional pre-processing steps to the input pipeline. Such solutions either have a hard time generalizing, require knowledge of adversarial attacks during training, or are computationally undesirable. Instead, we propose a framework called ELYTRA to take insights for parameter-efficient fine-tuning and use low-rank adaptation (LoRA) to train a lightweight security patch (or patches), enabling us to dynamically patch large pre-existing vision systems as new vulnerabilities are discovered. We demonstrate that the ELYTRA framework can patch pre-trained large vision models to improve classification accuracy by up to 24.09% in the presence of adversarial examples.