Boosting Privately: Privacy-Preserving Federated Extreme Boosting for Mobile Crowdsensing

TL;DR

FedXGB introduces a privacy-preserving federated XGBoost scheme that enhances security and efficiency in mobile crowdsensing by combining secret sharing and homomorphic encryption, with minimal accuracy loss.

Contribution

The paper presents a novel secure aggregation scheme for federated XGBoost that supports forced aggregation and robustness to user dropout, advancing privacy-preserving machine learning.

Findings

Achieves less than 1% accuracy loss compared to original XGBoost.

Reduces runtime by approximately 23.9%.

Reduces communication overhead by about 33.3%.

Abstract

Recently, Google and other 24 institutions proposed a series of open challenges towards federated learning (FL), which include application expansion and homomorphic encryption (HE). The former aims to expand the applicable machine learning models of FL. The latter focuses on who holds the secret key when applying HE to FL. For the naive HE scheme, the server is set to master the secret key. Such a setting causes a serious problem that if the server does not conduct aggregation before decryption, a chance is left for the server to access the user's update. Inspired by the two challenges, we propose FedXGB, a federated extreme gradient boosting (XGBoost) scheme supporting forced aggregation. FedXGB mainly achieves the following two breakthroughs. First, FedXGB involves a new HE based secure aggregation scheme for FL. By combining the advantages of secret sharing and homomorphic encryption, the algorithm can solve the second challenge mentioned above, and is robust to the user dropout. Then, FedXGB extends FL to a new machine learning model by applying the secure aggregation scheme to the classification and regression tree building of XGBoost. Moreover, we conduct a comprehensive theoretical analysis and extensive experiments to evaluate the security, effectiveness, and efficiency of FedXGB. The results indicate that FedXGB achieves less than 1% accuracy loss compared with the original XGBoost, and can provide about 23.9% runtime and 33.3% communication reduction for HE based model update aggregation of FL.