Network On Network for Tabular Data Classification in Real-world Applications

TL;DR

This paper introduces Network On Network (NON), a deep neural network model designed for tabular data classification that effectively captures intra-field information and non-linear feature interactions, outperforming existing methods.

Contribution

NON is a novel model that leverages a three-component architecture to fully exploit intra-field information and non-linear interactions in tabular data classification.

Findings

NON outperforms state-of-the-art models on six real-world datasets.

NON effectively captures intra-field information in embeddings.

Qualitative analysis shows improved modeling of feature interactions.

Abstract

Tabular data is the most common data format adopted by our customers ranging from retail, finance to E-commerce, and tabular data classification plays an essential role to their businesses. In this paper, we present Network On Network (NON), a practical tabular data classification model based on deep neural network to provide accurate predictions. Various deep methods have been proposed and promising progress has been made. However, most of them use operations like neural network and factorization machines to fuse the embeddings of different features directly, and linearly combine the outputs of those operations to get the final prediction. As a result, the intra-field information and the non-linear interactions between those operations (e.g. neural network and factorization machines) are ignored. Intra-field information is the information that features inside each field belong to the same field. NON is proposed to take full advantage of intra-field information and non-linear interactions. It consists of three components: field-wise network at the bottom to capture the intra-field information, across field network in the middle to choose suitable operations data-drivenly, and operation fusion network on the top to fuse outputs of the chosen operations deeply. Extensive experiments on six real-world datasets demonstrate NON can outperform the state-of-the-art models significantly. Furthermore, both qualitative and quantitative study of the features in the embedding space show NON can capture intra-field information effectively.