DecisioNet: A Binary-Tree Structured Neural Network

TL;DR

DecisioNet is a novel binary-tree structured neural network that transforms existing deep neural networks into lightweight models, combining neural learning with efficient inference to reduce computational costs without sacrificing accuracy.

Contribution

We introduce DecisioNet, a systematic method to convert DNNs into binary-tree structured networks, achieving efficiency gains while maintaining performance.

Findings

DecisioNet variants match baseline accuracy on datasets.

DecisioNet significantly reduces computational cost.

The approach is effective across multiple datasets.

Abstract

Deep neural networks (DNNs) and decision trees (DTs) are both state-of-the-art classifiers. DNNs perform well due to their representational learning capabilities, while DTs are computationally efficient as they perform inference along one route (root-to-leaf) that is dependent on the input data. In this paper, we present DecisioNet (DN), a binary-tree structured neural network. We propose a systematic way to convert an existing DNN into a DN to create a lightweight version of the original model. DecisioNet takes the best of both worlds - it uses neural modules to perform representational learning and utilizes its tree structure to perform only a portion of the computations. We evaluate various DN architectures, along with their corresponding baseline models on the FashionMNIST, CIFAR10, and CIFAR100 datasets. We show that the DN variants achieve similar accuracy while significantly reducing the computational cost of the original network.