Enhancing Temporal Awareness in LLMs for Temporal Point Processes

TL;DR

This paper introduces TPP-TAL, a framework that enhances large language models' ability to understand and model temporal dependencies in event sequences, significantly improving their performance on temporal point process tasks.

Contribution

The paper proposes a novel plug-and-play method that explicitly aligns temporal dynamics with semantic context in LLMs, improving their temporal reasoning capabilities.

Findings

Significant improvements in event prediction accuracy.

Enhanced temporal likelihood estimation.

Better modeling of long-range temporal dependencies.

Abstract

Temporal point processes (TPPs) are crucial for analyzing events over time and are widely used in fields such as finance, healthcare, and social systems. These processes are particularly valuable for understanding how events unfold over time, accounting for their irregularity and dependencies. Despite the success of large language models (LLMs) in sequence modeling, applying them to temporal point processes remains challenging. A key issue is that current methods struggle to effectively capture the complex interaction between temporal information and semantic context, which is vital for accurate event modeling. In this context, we introduce TPP-TAL (Temporal Point Processes with Enhanced Temporal Awareness in LLMs), a novel plug-and-play framework designed to enhance temporal reasoning within LLMs. Rather than using the conventional method of simply concatenating event time and type embeddings, TPP-TAL explicitly aligns temporal dynamics with contextual semantics before feeding this information into the LLM. This alignment allows the model to better perceive temporal dependencies and long-range interactions between events and their surrounding contexts. Through comprehensive experiments on several benchmark datasets, it is shown that TPP-TAL delivers substantial improvements in temporal likelihood estimation and event prediction accuracy, highlighting the importance of enhancing temporal awareness in LLMs for continuous-time event modeling. The code is made available at https://github.com/chenlilil/TPP-TAL