Attributed Sequence Embedding

TL;DR

This paper introduces NAS, a deep multimodal learning framework for unsupervised embedding of attributed sequences, effectively capturing complex dependencies for diverse data mining tasks.

Contribution

It presents the first deep learning approach specifically designed for attributed sequence embedding, handling heterogeneous items and attributes in an unsupervised manner.

Findings

Embeddings improve performance on clustering tasks.

Effective across various real-world datasets.

Unsupervised approach generalizes well to different applications.

Abstract

Mining tasks over sequential data, such as clickstreams and gene sequences, require a careful design of embeddings usable by learning algorithms. Recent research in feature learning has been extended to sequential data, where each instance consists of a sequence of heterogeneous items with a variable length. However, many real-world applications often involve attributed sequences, where each instance is composed of both a sequence of categorical items and a set of attributes. In this paper, we study this new problem of attributed sequence embedding, where the goal is to learn the representations of attributed sequences in an unsupervised fashion. This problem is core to many important data mining tasks ranging from user behavior analysis to the clustering of gene sequences. This problem is challenging due to the dependencies between sequences and their associated attributes. We propose a deep multimodal learning framework, called NAS, to produce embeddings of attributed sequences. The embeddings are task independent and can be used on various mining tasks of attributed sequences. We demonstrate the effectiveness of our embeddings of attributed sequences in various unsupervised learning tasks on real-world datasets.