Osmosis Distillation: Model Hijacking with the Fewest Samples

TL;DR

This paper introduces Osmosis Distillation, a novel attack method that hijacks models using very few poisoned synthetic samples, highlighting security risks in transfer learning with dataset distillation.

Contribution

The paper presents the first model hijacking attack leveraging synthetic datasets from dataset distillation, demonstrating high success rates with minimal samples across various models.

Findings

High attack success rates in hidden tasks

Preserves model utility in original tasks

Effective across diverse model architectures

Abstract

Transfer learning is devised to leverage knowledge from pre-trained models to solve new tasks with limited data and computational resources. Meanwhile, dataset distillation has emerged to synthesize a compact dataset that preserves critical information from the original large dataset. Therefore, a combination of transfer learning and dataset distillation offers promising performance in evaluations. However, a non-negligible security threat remains undiscovered in transfer learning using synthetic datasets generated by dataset distillation methods, where an adversary can perform a model hijacking attack with only a few poisoned samples in the synthetic dataset. To reveal this threat, we propose Osmosis Distillation (OD) attack, a novel model hijacking strategy that targets deep learning models using the fewest samples. Comprehensive evaluations on various datasets demonstrate that the OD attack attains high attack success rates in hidden tasks while preserving high model utility in original tasks. Furthermore, the distilled osmosis set enables model hijacking across diverse model architectures, allowing model hijacking in transfer learning with considerable attack performance and model utility. We argue that awareness of using third-party synthetic datasets in transfer learning must be raised.