Black-Box Privacy Attacks on Shared Representations in Multitask Learning

TL;DR

This paper reveals that shared representations in multitask learning can unintentionally leak sensitive task information through black-box inference attacks, even without access to training data, highlighting privacy vulnerabilities.

Contribution

It introduces a novel black-box attack model for inferring task membership from shared representations in multitask learning, with theoretical and empirical validation across domains.

Findings

Black-box attacks can successfully infer task membership from shared representations.

Even with only fresh samples, adversaries can breach privacy without access to training data.

Theoretical analysis shows a clear separation between training and fresh sample adversaries.

Abstract

Multitask learning (MTL) has emerged as a powerful paradigm that leverages similarities among multiple learning tasks, each with insufficient samples to train a standalone model, to solve them simultaneously while minimizing data sharing across users and organizations. MTL typically accomplishes this goal by learning a shared representation that captures common structure among the tasks by embedding data from all tasks into a common feature space. Despite being designed to be the smallest unit of shared information necessary to effectively learn patterns across multiple tasks, these shared representations can inadvertently leak sensitive information about the particular tasks they were trained on. In this work, we investigate what information is revealed by the shared representations through the lens of inference attacks. Towards this, we propose a novel, black-box task-inference threat model where the adversary, given the embedding vectors produced by querying the shared representation on samples from a particular task, aims to determine whether that task was present when training the shared representation. We develop efficient, purely black-box attacks on machine learning models that exploit the dependencies between embeddings from the same task without requiring shadow models or labeled reference data. We evaluate our attacks across vision and language domains for multiple use cases of MTL and demonstrate that even with access only to fresh task samples rather than training data, a black-box adversary can successfully infer a task's inclusion in training. To complement our experiments, we provide theoretical analysis of a simplified learning setting and show a strict separation between adversaries with training samples and fresh samples from the target task's distribution.