Deep Neural Decision Trees

TL;DR

This paper introduces Deep Neural Decision Trees (DNDT), a novel model combining the interpretability of decision trees with the training advantages of neural networks, optimized via gradient descent for tabular data.

Contribution

The work presents DNDT, a neural network-based decision tree model that is inherently interpretable and trainable with gradient descent, unlike traditional greedy algorithms.

Findings

DNDT performs effectively on various tabular datasets.

DNDT naturally self-prunes at split and feature levels.

DNDT bridges the gap between neural networks and decision trees.

Abstract

Deep neural networks have been proven powerful at processing perceptual data, such as images and audio. However for tabular data, tree-based models are more popular. A nice property of tree-based models is their natural interpretability. In this work, we present Deep Neural Decision Trees (DNDT) -- tree models realised by neural networks. A DNDT is intrinsically interpretable, as it is a tree. Yet as it is also a neural network (NN), it can be easily implemented in NN toolkits, and trained with gradient descent rather than greedy splitting. We evaluate DNDT on several tabular datasets, verify its efficacy, and investigate similarities and differences between DNDT and vanilla decision trees. Interestingly, DNDT self-prunes at both split and feature-level.