SilentWood: Private Inference Over Gradient-Boosting Decision Forests

TL;DR

SilentWood introduces an efficient private inference protocol for gradient boosting decision forests, significantly reducing computation and communication costs while preserving data and model privacy, enabling scalable secure machine learning.

Contribution

It is the first private inference protocol tailored for scalable gradient boosting decision forests, with several optimizations to improve efficiency over naive methods.

Findings

Inference time up to 42.5x faster than baseline

Communication and computation costs significantly reduced

First scalable private inference protocol for gradient boosting forests

Abstract

Gradient boosting decision forests, used by XGBoost or AdaBoost, offer higher accuracy and lower training times than decision trees for large datasets. Protocols for private inference over decision trees can be used to preserve the privacy of the input data as well as the privacy of the trees. However, naively extending private inference over decision trees to private inference over decision forests by replicating the protocols leads to impractical running times. In this paper, we propose an efficient private decision inference protocol using homomorphic encryption. We present several optimizations that identify and then remove (approximate) duplication between the trees in a forest, thereby achieving significant improvements in communication and computation cost over the naive approach. To the best of our knowledge, we present the first private inference protocol for highly scalable gradient boosting decision forests. Our protocol's (SilentWood) inference time is faster than the baseline of parallel running the RCC-PDTE protocol by Mahdavi et al. by up to 42.5x, and faster than Zama's Concrete ML XGBoost by up to 27.8x, and faster than SoK-GGG's two-party garbled circuit protocol by 2.94x.