SimCURL: Simple Contrastive User Representation Learning from Command Sequences

TL;DR

SimCURL is a contrastive self-supervised learning framework that effectively learns user representations from unlabeled command sequences, improving downstream user modeling tasks.

Contribution

It introduces a novel user-session network architecture and session dropout data augmentation for learning from unlabeled command data.

Findings

Significant improvement over existing methods in downstream tasks.

Effective learning from large-scale unlabeled command sequences.

Demonstrated on a dataset with over half a billion commands.

Abstract

User modeling is crucial to understanding user behavior and essential for improving user experience and personalized recommendations. When users interact with software, vast amounts of command sequences are generated through logging and analytics systems. These command sequences contain clues to the users' goals and intents. However, these data modalities are highly unstructured and unlabeled, making it difficult for standard predictive systems to learn from. We propose SimCURL, a simple yet effective contrastive self-supervised deep learning framework that learns user representation from unlabeled command sequences. Our method introduces a user-session network architecture, as well as session dropout as a novel way of data augmentation. We train and evaluate our method on a real-world command sequence dataset of more than half a billion commands. Our method shows significant improvement over existing methods when the learned representation is transferred to downstream tasks such as experience and expertise classification.