Classification Models for Partially Ordered Sequences

TL;DR

This paper introduces a transformer-based model for classifying partially-ordered sequences, outperforming existing set models and leveraging transition probabilities to enhance prediction accuracy.

Contribution

The paper presents a novel transformer model tailored for partially-ordered sequence classification and benchmarks it against existing order-invariant models.

Findings

Transformer model outperforms existing set models on three datasets.

Transition probabilities improve model performance.

Model effectively handles data with uncertain or granular timestamps.

Abstract

Many models such as Long Short Term Memory (LSTMs), Gated Recurrent Units (GRUs) and transformers have been developed to classify time series data with the assumption that events in a sequence are ordered. On the other hand, fewer models have been developed for set based inputs, where order does not matter. There are several use cases where data is given as partially-ordered sequences because of the granularity or uncertainty of time stamps. We introduce a novel transformer based model for such prediction tasks, and benchmark against extensions of existing order invariant models. We also discuss how transition probabilities between events in a sequence can be used to improve model performance. We show that the transformer-based equal-time model outperforms extensions of existing set models on three data sets.