LLM4ES: Learning User Embeddings from Event Sequences via Large Language Models

TL;DR

LLM4ES leverages large language models to generate high-quality user embeddings from event sequences, enhancing user classification and enabling diverse applications across finance and healthcare.

Contribution

The paper introduces a novel framework that transforms event sequences into textual data for LLM fine-tuning, with a text enrichment technique to improve embeddings in low-variability domains.

Findings

Achieves state-of-the-art user classification performance

Outperforms existing embedding methods

Effective in finance and healthcare applications

Abstract

This paper presents LLM4ES, a novel framework that exploits large pre-trained language models (LLMs) to derive user embeddings from event sequences. Event sequences are transformed into a textual representation, which is subsequently used to fine-tune an LLM through next-token prediction to generate high-quality embeddings. We introduce a text enrichment technique that enhances LLM adaptation to event sequence data, improving representation quality for low-variability domains. Experimental results demonstrate that LLM4ES achieves state-of-the-art performance in user classification tasks in financial and other domains, outperforming existing embedding methods. The resulting user embeddings can be incorporated into a wide range of applications, from user segmentation in finance to patient outcome prediction in healthcare.