Private Boosted Decision Trees via Smooth Re-Weighting

TL;DR

This paper introduces a differentially private boosting algorithm for decision trees that maintains individual privacy by limiting data influence, resulting in improved model sparsity and accuracy over existing private ensemble methods.

Contribution

It presents a novel boosting method that guarantees differential privacy by smooth re-weighting, enhancing model performance while protecting individual data privacy.

Findings

Better model sparsity compared to existing methods

Higher accuracy in private ensemble classifiers

Effective privacy preservation through weight control

Abstract

Protecting the privacy of people whose data is used by machine learning algorithms is important. Differential Privacy is the appropriate mathematical framework for formal guarantees of privacy, and boosted decision trees are a popular machine learning technique. So we propose and test a practical algorithm for boosting decision trees that guarantees differential privacy. Privacy is enforced because our booster never puts too much weight on any one example; this ensures that each individual's data never influences a single tree "too much." Experiments show that this boosting algorithm can produce better model sparsity and accuracy than other differentially private ensemble classifiers.