Effective and Efficient Federated Tree Learning on Hybrid Data

TL;DR

This paper introduces HybridTree, a federated learning method for hybrid data that efficiently builds decision trees across parties with different features and samples, achieving high accuracy with low communication overhead.

Contribution

HybridTree is the first federated tree learning approach designed for hybrid data, incorporating theoretical insights and a layer-level training solution to reduce communication.

Findings

Achieves comparable accuracy to centralized models.

Up to 8 times faster than baseline methods.

Requires minimal communication overhead.

Abstract

Federated learning has emerged as a promising distributed learning paradigm that facilitates collaborative learning among multiple parties without transferring raw data. However, most existing federated learning studies focus on either horizontal or vertical data settings, where the data of different parties are assumed to be from the same feature or sample space. In practice, a common scenario is the hybrid data setting, where data from different parties may differ both in the features and samples. To address this, we propose HybridTree, a novel federated learning approach that enables federated tree learning on hybrid data. We observe the existence of consistent split rules in trees. With the help of these split rules, we theoretically show that the knowledge of parties can be incorporated into the lower layers of a tree. Based on our theoretical analysis, we propose a layer-level solution that does not need frequent communication traffic to train a tree. Our experiments demonstrate that HybridTree can achieve comparable accuracy to the centralized setting with low computational and communication overhead. HybridTree can achieve up to 8 times speedup compared with the other baselines.