A Survey of Neural Trees

TL;DR

This survey comprehensively reviews neural trees, a hybrid model integrating neural networks and decision trees, highlighting their interpretability benefits, taxonomy, challenges, and future directions in machine learning.

Contribution

It provides the first thorough taxonomy of neural trees and analyzes their interpretability and performance, guiding future research in this hybrid model domain.

Findings

Neural trees enhance interpretability compared to traditional neural networks.

Different types of neural trees show varied performance and interpretability trade-offs.

The survey identifies key challenges and potential solutions for neural tree development.

Abstract

Neural networks (NNs) and decision trees (DTs) are both popular models of machine learning, yet coming with mutually exclusive advantages and limitations. To bring the best of the two worlds, a variety of approaches are proposed to integrate NNs and DTs explicitly or implicitly. In this survey, these approaches are organized in a school which we term as neural trees (NTs). This survey aims to present a comprehensive review of NTs and attempts to identify how they enhance the model interpretability. We first propose a thorough taxonomy of NTs that expresses the gradual integration and co-evolution of NNs and DTs. Afterward, we analyze NTs in terms of their interpretability and performance, and suggest possible solutions to the remaining challenges. Finally, this survey concludes with a discussion about other considerations like conditional computation and promising directions towards this field. A list of papers reviewed in this survey, along with their corresponding codes, is available at: https://github.com/zju-vipa/awesome-neural-trees