Vector Embeddings with Subvector Permutation Invariance using a Triplet Enhanced Autoencoder

TL;DR

This paper introduces a triplet loss-enhanced autoencoder that produces vector embeddings nearly invariant to subvector permutations, improving clustering and classification tasks involving permuted data.

Contribution

The paper proposes a novel autoencoder architecture with triplet loss to achieve permutation-invariant embeddings for subvectors, enhancing data analysis tasks.

Findings

Embeddings are nearly invariant to subvector permutations.

Improved clustering and classification accuracy with permutation-invariant embeddings.

Autoencoder effectively captures subtle data patterns.

Abstract

The use of deep neural network (DNN) autoencoders (AEs) has recently exploded due to their wide applicability. However, the embedding representation produced by a standard DNN AE that is trained to minimize only the reconstruction error does not always reveal more subtle patterns in the data. Sometimes, the autoencoder needs further direction in the form of one or more additional loss functions. In this paper, we use an autoencoder enhanced with triplet loss to promote the clustering of vectors that are related through permutations of constituent subvectors. With this approach, we can create an embedding of the vector that is nearly invariant to such permutations. We can then use these invariant embeddings as inputs to other problems, like classification and clustering, and improve detection accuracy in those problems.