Group-Connected Multilayer Perceptron Networks

TL;DR

This paper introduces Group-Connected Multilayer Perceptron (GMLP) networks that learn feature groups for deep representation in unstructured domains, achieving state-of-the-art results on various datasets.

Contribution

GMLP is a novel architecture that learns sparse feature groups and reduces network complexity through local group-wise operations and a tree-like structure.

Findings

GMLP successfully learns expressive feature combinations.

Achieves state-of-the-art classification performance.

Effectively visualizes feature groupings on datasets.

Abstract

Despite the success of deep learning in domains such as image, voice, and graphs, there has been little progress in deep representation learning for domains without a known structure between features. For instance, a tabular dataset of different demographic and clinical factors where the feature interactions are not given as a prior. In this paper, we propose Group-Connected Multilayer Perceptron (GMLP) networks to enable deep representation learning in these domains. GMLP is based on the idea of learning expressive feature combinations (groups) and exploiting them to reduce the network complexity by defining local group-wise operations. During the training phase, GMLP learns a sparse feature grouping matrix using temperature annealing softmax with an added entropy loss term to encourage the sparsity. Furthermore, an architecture is suggested which resembles binary trees, where group-wise operations are followed by pooling operations to combine information; reducing the number of groups as the network grows in depth. To evaluate the proposed method, we conducted experiments on different real-world datasets covering various application areas. Additionally, we provide visualizations on MNIST and synthesized data. According to the results, GMLP is able to successfully learn and exploit expressive feature combinations and achieve state-of-the-art classification performance on different datasets.