Deep Variable-Block Chain with Adaptive Variable Selection

TL;DR

This paper introduces Deep Variable-Block Chain (DVC), a neural network framework that constructs variable blocks with chain structures for improved high-dimensional data classification, adaptive variable selection, and interpretability.

Contribution

The paper proposes a novel DNN architecture that uses step-wise greedy search to form variable blocks with chain structures and incorporates adaptive variable selection based on decision regions.

Findings

DVC outperforms traditional DNNs and classifiers in accuracy.

DVC achieves high accuracy with reduced dimensionality.

DVC identifies different relevant variables across regions.

Abstract

The architectures of deep neural networks (DNN) rely heavily on the underlying grid structure of variables, for instance, the lattice of pixels in an image. For general high dimensional data with variables not associated with a grid, the multi-layer perceptron and deep belief network are often used. However, it is frequently observed that those networks do not perform competitively and they are not helpful for identifying important variables. In this paper, we propose a framework that imposes on blocks of variables a chain structure obtained by step-wise greedy search so that the DNN architecture can leverage the constructed grid. We call this new neural network Deep Variable-Block Chain (DVC). Because the variable blocks are used for classification in a sequential manner, we further develop the capacity of selecting variables adaptively according to a number of regions trained by a decision tree. Our experiments show that DVC outperforms other generic DNNs and other strong classifiers. Moreover, DVC can achieve high accuracy at much reduced dimensionality and sometimes reveals drastically different sets of relevant variables for different regions.