Distributed Decision Trees

TL;DR

This paper introduces distributed decision trees that enable multiple paths to be traversed simultaneously, combining the interpretability of trees with the distributed representation power of neural networks, and demonstrates their competitive performance.

Contribution

The paper extends budding trees to distributed trees with independent splits, allowing multiple paths to be active simultaneously, enhancing representation capacity.

Findings

Distributed trees perform comparably or better than budding trees.

Distributed trees enable multiple path traversal at once.

They are effective for classification and regression tasks.

Abstract

Recently proposed budding tree is a decision tree algorithm in which every node is part internal node and part leaf. This allows representing every decision tree in a continuous parameter space, and therefore a budding tree can be jointly trained with backpropagation, like a neural network. Even though this continuity allows it to be used in hierarchical representation learning, the learned representations are local: Activation makes a soft selection among all root-to-leaf paths in a tree. In this work we extend the budding tree and propose the distributed tree where the children use different and independent splits and hence multiple paths in a tree can be traversed at the same time. This ability to combine multiple paths gives the power of a distributed representation, as in a traditional perceptron layer. We show that distributed trees perform comparably or better than budding and traditional hard trees on classification and regression tasks.