Transfer Learning without Knowing: Reprogramming Black-box Machine Learning Models with Scarce Data and Limited Resources

TL;DR

This paper introduces black-box adversarial reprogramming (BAR), a novel method to repurpose unknown ML models for new tasks using minimal data and resources, without needing model details.

Contribution

BAR enables transfer learning by manipulating black-box models solely through input-output responses, outperforming existing methods in scarce data scenarios.

Findings

BAR outperforms state-of-the-art methods in medical diagnosis tasks.

BAR achieves comparable performance to full-knowledge adversarial reprogramming.

BAR significantly surpasses baseline transfer learning approaches.

Abstract

Current transfer learning methods are mainly based on finetuning a pretrained model with target-domain data. Motivated by the techniques from adversarial machine learning (ML) that are capable of manipulating the model prediction via data perturbations, in this paper we propose a novel approach, black-box adversarial reprogramming (BAR), that repurposes a well-trained black-box ML model (e.g., a prediction API or a proprietary software) for solving different ML tasks, especially in the scenario with scarce data and constrained resources. The rationale lies in exploiting high-performance but unknown ML models to gain learning capability for transfer learning. Using zeroth order optimization and multi-label mapping techniques, BAR can reprogram a black-box ML model solely based on its input-output responses without knowing the model architecture or changing any parameter. More importantly, in the limited medical data setting, on autism spectrum disorder classification, diabetic retinopathy detection, and melanoma detection tasks, BAR outperforms state-of-the-art methods and yields comparable performance to the vanilla adversarial reprogramming method requiring complete knowledge of the target ML model. BAR also outperforms baseline transfer learning approaches by a significant margin, demonstrating cost-effective means and new insights for transfer learning.