Privacy in Metalearning and Multitask Learning: Modeling and Separations

TL;DR

This paper systematically studies privacy in personalized machine learning, especially multitask and metalearning, by creating a taxonomy and proving fundamental separations between different private learning frameworks.

Contribution

It introduces a taxonomy of formal frameworks for private personalized learning and establishes novel theoretical separations between private multitask learning and metalearning.

Findings

Constructed a taxonomy of formal frameworks for private personalized learning.

Proved a novel separation between private multitask learning and private metalearning.

Abstract

Model personalization allows a set of individuals, each facing a different learning task, to train models that are more accurate for each person than those they could develop individually. The goals of personalization are captured in a variety of formal frameworks, such as multitask learning and metalearning. Combining data for model personalization poses risks for privacy because the output of an individual's model can depend on the data of other individuals. In this work we undertake a systematic study of differentially private personalized learning. Our first main contribution is to construct a taxonomy of formal frameworks for private personalized learning. This taxonomy captures different formal frameworks for learning as well as different threat models for the attacker. Our second main contribution is to prove separations between the personalized learning problems corresponding to different choices. In particular, we prove a novel separation between private multitask learning and private metalearning.