Querying Kernel Methods Suffices for Reconstructing their Training Data

TL;DR

This paper demonstrates that kernel methods can be reconstructed from model outputs alone, raising privacy concerns about data memorization even without access to model parameters.

Contribution

It provides both empirical and theoretical evidence that querying kernel models can reveal their training data, highlighting privacy risks.

Findings

Kernel models can be reconstructed from outputs alone.

Reconstruction applies to various kernel methods.

Results suggest privacy vulnerabilities in kernel models.

Abstract

Over-parameterized models have raised concerns about their potential to memorize training data, even when achieving strong generalization. The privacy implications of such memorization are generally unclear, particularly in scenarios where only model outputs are accessible. We study this question in the context of kernel methods, and demonstrate both empirically and theoretically that querying kernel models at various points suffices to reconstruct their training data, even without access to model parameters. Our results hold for a range of kernel methods, including kernel regression, support vector machines, and kernel density estimation. Our hope is that this work can illuminate potential privacy concerns for such models.