Classifying Unordered Feature Sets with Convolutional Deep Averaging Networks

TL;DR

This paper introduces convolutional deep averaging networks (CDANs), a novel neural network architecture designed to classify and learn from unordered, variable-sized feature sets while maintaining permutation invariance.

Contribution

The paper presents CDANs as an efficient, permutation-invariant neural network architecture that effectively handles unordered, variable-size feature sets with nonlinear embeddings.

Findings

CDANs outperform linear embedding methods in experiments.

Nonlinear embeddings significantly improve classification accuracy.

CDANs are computationally efficient and flexible for variable-sized sets.

Abstract

Unordered feature sets are a nonstandard data structure that traditional neural networks are incapable of addressing in a principled manner. Providing a concatenation of features in an arbitrary order may lead to the learning of spurious patterns or biases that do not actually exist. Another complication is introduced if the number of features varies between each set. We propose convolutional deep averaging networks (CDANs) for classifying and learning representations of datasets whose instances comprise variable-size, unordered feature sets. CDANs are efficient, permutation-invariant, and capable of accepting sets of arbitrary size. We emphasize the importance of nonlinear feature embeddings for obtaining effective CDAN classifiers and illustrate their advantages in experiments versus linear embeddings and alternative permutation-invariant and -equivariant architectures.