Learning to Obstruct Few-Shot Image Classification over Restricted Classes

TL;DR

This paper introduces a meta-learning approach called Learning to Obstruct (LTO) that makes few-shot classification on restricted classes more difficult, aiming to prevent misuse of pre-trained models for harmful tasks while preserving overall performance.

Contribution

The paper proposes a novel meta-learning method to intentionally hinder fine-tuning on specific classes without affecting other classes' performance.

Findings

LTO successfully obstructs four FSC methods across multiple datasets.

LTO maintains performance on non-restricted classes.

The approach demonstrates potential for safer deployment of pre-trained models.

Abstract

Advancements in open-source pre-trained backbones make it relatively easy to fine-tune a model for new tasks. However, this lowered entry barrier poses potential risks, e.g., bad actors developing models for harmful applications. A question arises: Is possible to develop a pre-trained model that is difficult to fine-tune for certain downstream tasks? To begin studying this, we focus on few-shot classification (FSC). Specifically, we investigate methods to make FSC more challenging for a set of restricted classes while maintaining the performance of other classes. We propose to meta-learn over the pre-trained backbone in a manner that renders it a ''poor initialization''. Our proposed Learning to Obstruct (LTO) algorithm successfully obstructs four FSC methods across three datasets, including ImageNet and CIFAR100 for image classification, as well as CelebA for attribute classification.